acceptodds
Under review as a conference paper at ICLR 2027

FocusMe: Bone-Conduction-Guided Speech Enhancement in Challenging Acoustic Scenarios

Abstract

Recovering a wearer's speech from an air-conduction (AC) mixture requires both suppressing interference and identifying the target speaker, particularly when the interference includes other voices. We propose FocusMe, a causal speech enhancement system that uses synchronized bone-conduction (BC) speech for two complementary purposes: acoustic reconstruction and enrollment-free speaker conditioning. Cross-Attention Feature Fusion retrieves BC information relevant to the AC mixture and adaptively combines the two signals, while a causal extractor supplies frame-level BC speaker representations to the reconstruction network. The system is trained jointly for speech reconstruction and speaker classification. Across 13 test conditions constructed from ABCS and ESMB, FocusMe achieves the best average perceptual quality, intelligibility, and recognition accuracy among the compared systems, reducing mean character error rate from 22.85% for DCCRN to 17.03%. These results demonstrate the effectiveness of jointly exploiting BC speech for acoustic reconstruction and speaker conditioning under severe noise and competing speech.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
bone conduction speech, multimodal speech enhancement, multi-task learning, speaker representation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_737KOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FocusMe, a speech enhancement framework that combines air-conduction and bone-conduction signals to recover target speech under environmental noise and competing speech. The method introduces two components: (1) Cross-Attention Feature Fusion (CAFF), which adaptively combines AC and BC spectral features, and (2) a Frame-Level Speaker Embedding (FSE) extractor, which derives speaker-discriminative representations from synchronized BC speech without requiring a separate enrollment utterance. The model is jointly trained using speech reconstruction and speaker classification objectives.

Critical strengths
  1. The paper addresses an important problem: speech enhancement in challenging environments containing both environmental noise and competing speakers. Using synchronized BC signals for target speech extraction without separate speaker enrollment is appealing. The proposed framework incorporates BC information for both acoustic reconstruction and speaker conditioning. Explicitly supervising speaker-discriminative representations alongside reconstruction to improve target speech preservation.

  2. FocusMe achieves the best average PESQ, STOI, ESTOI, DNSMOS, and CER among the compared methods. Improvements are also observed under severe environmental noise and competing speech.

Critical weaknesses
  1. The FSE extractor is intended to provide speaker conditioning, but the embeddings are derived from synchronized BC speech, which contains both speaker characteristics and phonetic information aligned with the target AC signal. So, it is unclear whether the improvements arise primarily from better speaker identification or from additional synchronized speech information. Although removing the speaker classification objective degrades performance, this does not fully explain the learned representations.

  2. The experiments introduce environmental noise and competing speech into AC recordings while retaining the corresponding BC signals without additional corruption. However, BC sensors in wearable devices can experience motion artifacts, contact variations, and mechanical vibrations from activities such as walking or head movement. Since FocusMe relies on BC signals for both acoustic fusion and speaker conditioning, such distortions could affect its performance. Although the authors acknowledge this limitation, the current evaluation does not establish how robust the proposed method is under realistic wearable conditions.

  3. The evaluation uses synthetically constructed AC mixtures from synchronized ABCS and ESMB recordings. Although the speaker-disjoint splits are good, the experiments do not evaluate recordings captured in genuinely noisy, reverberant environments with simultaneous AC and BC interference. Furthermore, differences in BC sensor placement and contact characteristics across wearable devices could affect the learned representations. These limitations question the method's application to real-world settings.

Questions for authors
  1. Can the authors provide additional insight into whether the FSE representations primarily encode speaker identity or synchronized phonetic information? For example, comparing embeddings from synchronized BC speech with embeddings extracted from a different utterance by the same speaker could help distinguish these contributions.

  2. How sensitive is FocusMe to BC signal degradation, such as motion artifacts, contact variations, or temporary loss of sensor contact? Even a limited sensitivity analysis using synthetic BC perturbations would help assess the reliability of the proposed dual-use BC conditioning mechanism.

  3. It would be valuable to evaluate FocusMe on real-world noisy AC–BC recordings and across different BC sensor configurations to establish whether the reported improvements generalize beyond the controlled setups.

1 comment

Q1: Can the authors provide additional insight into whether the FSE representations primarily encode speaker identity or synchronized phonetic information?

A1: We thank the reviewers for pointing this out. We would like to clarify that Section 5.4 analyzes frame-level FSE embeddings from three speakers excluded from training, with 100 utterances per speaker. PCA projections show that embeddings from the same speaker tend to form compact clusters, and K-means assignments largely align with speaker labels. These findings provide qualitative evidence that FSE retains speaker-discriminative information across utterances. The ablation results further support FSE’s contribution to FocusMe’s enhancement performance.

Q2: How sensitive is FocusMe to BC signal degradation, such as motion artifacts, contact variations, or temporary loss of sensor contact?

A2: Following the reviewer’s suggestion, we recorded speech under six conditions to assess FocusMe’s sensitivity to practical variations in headset use: (a) normal wearing, (b) head movement, (c) walking, (d) touching the headset by hand, (e) walking while only the interfering speaker was speaking, and (f) loose wearing. We have added this study to the appendix of the revised manuscript. The table below reports the results for conditions (a)–(d) (Table 9, Appendix E.3). ΔWER denotes the change relative to condition (a), measured in percentage points (pp).

ConditionTarget only: WER (%) ↓Target only: ΔWER (pp)Target + interfering speech: WER (%) ↓Target + interfering speech: ΔWER (pp)
(a) Normal wearing1.070.004.300.00
(b) Head movement2.86+1.796.81+2.51
(c) Walking0.71−0.363.22−1.08
(d) Touching the headset2.15+1.084.65+0.35

The results indicate that FocusMe tolerates the tested movement and contact perturbations. The qualitative example for condition (e) also illustrates effective suppression of interfering speech while the wearer is walking (Figure 10, Appendix E.3). However, a loose headset fit remains a failure case: FocusMe and all evaluated baselines perform poorly under this condition (Figure 11, Appendix E.3). Improving robustness to variations in headset contact is therefore an important direction for future work.

Q3: It would be valuable to evaluate FocusMe on real-world noisy AC–BC recordings and across different BC sensor configurations to establish whether the reported improvements generalize beyond the controlled setups.

A3: To address this suggestion, we recorded paired AC–BC signals directly from a headset in a subway and evaluated FocusMe and the baseline methods on the same recorded segment. We have added this study to the appendix of the revised manuscript. Appendix E.1 describes the recording setup, and Appendix E.2 presents the output spectrograms. For this recording, the spectrograms qualitatively suggest that FocusMe suppresses noise more effectively than the baselines while preserving more speech detail. Although EBEN [1], a BC-only system, reconstructs more mid- and high-frequency components, informal listening suggested distortions in speaker timbre and possible changes in the perceived speech content. Due to equipment constraints, we have not yet evaluated FocusMe’s generalization across different BC sensor configurations. This evaluation will be an important focus of our future work.

Reference

[1] Julien Hauret, Thomas Joubaud, Veronique Zimpfer, and ´´Eric Bavu. EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones. In IEEE International Conference on Acoustics, Speech and Signal Processing, pp.1–5, 2023.

Reviewer_5u8qOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper leverages synchronized bone-conduction (BC) signals to help a standard microphone recover the wearer’s own speech, even in environments with high levels of ambient noise or multiple simultaneous speakers. The system employs a Cross-Attention Feature Fusion (CAFF) module to integrate signals from the standard microphone and the bone-conduction sensor, while a Feature Extraction (FSE) module derives speaker-specific characteristics from the BC signal to assist in reconstructing the wearer’s voice. Experimental results demonstrate improvements in sound quality, intelligibility, and recognition performance.

Critical strengths
  1. The experimental comparison is fair. The authors retrained all baseline models included in the comparison, using consistent data splits and evaluating them on the same test data. Furthermore, there is no overlap between the speakers in the training and test sets.
Critical weaknesses
  1. The necessity of using both AC and BC simultaneously has not been sufficiently justified. The paper states that AC can supplement the high-frequency information missing from BC, yet it fails to demonstrate—through experimental evidence—the specific extent of the improvement gained by incorporating AC. Existing methods, such as EBEN, Aero and TRAMBA, are capable of reconstructing speech directly from BC signals; however, the paper does not provide a comparison against these BC-only approaches. Furthermore, in the experimental setup, noise and the speech of other individuals are introduced exclusively into the AC channel, leaving the BC channel free from such additional interference. Under these conditions, a comparison with methods that rely solely on BC is essential; otherwise, it is difficult to ascertain the actual contribution of AC to the final results.

  2. There is a lack of direct comparison with the relevant prior work. In the "Related Work" section, the paper mentions MBENet, which also utilizes both AC and BC for target speaker extraction, a task and input configuration highly similar to those of FocusMe. Nevertheless, the authors neither include MBENet as a baseline nor provide a justification for its omission.

Questions for authors
  1. Why is it necessary to use both AC and BC? Existing methods, such as EBEN, Aero and TRAMBA, are capable of reconstructing speech directly from BC signals. Although paper mentions that AC can supplement the high-frequency information missing from BC, it does not provide a comparison against BC-only approaches. In the experiments, specifically, noise and the voices of other speakers are added only to the AC signal, while the BC signal remains free from such additional interference. Could the authors clarify the extent of the performance gains—if any—achieved by incorporating AC under these conditions? Furthermore, would it be possible to provide a comparison against robust BC-only methods using the same dataset and testing conditions?

  2. Why is there no comparison with MBENet? Paper mentions MBENet in the "Related Work" section; it also utilizes AC and BC for target speaker extraction and is very similar to FocusMe in terms of both its task and input requirements. However, MBENet is not included in the experiments, and the authors provide no explanation for this omission. Could you clarify why MBENet was not selected as a baseline? Would it be possible to provide comparison results under the same experimental setup?

  3. Were multiple random seeds used in the experiments? The paper mentions the random seed 2666 in the appendix, but it appears this seed was used solely for dataset splitting. It remains unclear whether the reported results are derived from a single training run or from multiple runs using different random seeds. Could the authors clarify this and provide the mean and standard deviation of key metrics for FocusMe and the primary baseline methods, based on multiple independent training runs, to demonstrate the stability of the experimental results?

  4. How does the method perform in real-world noisy environments? The current experiments involve adding synthetic noise and speech from other speakers to "clean" AC audio, while the BC signals remain as originally recorded, without additional interference. In practical applications, however, BC signals can be affected by factors such as how the device is worn, variations in contact, or physical movement; furthermore, AC and BC signals may not always be perfectly synchronized. Do the authors have experimental results for these scenarios, or have the authors tested the method using AC–BC recordings captured in actual, noisy environments?

4 comments

Q1: Why is it necessary to use both AC and BC? Could the authors clarify the extent of the performance gains—if any—achieved by incorporating AC under these conditions? Furthermore, would it be possible to provide a comparison against robust BC-only methods using the same dataset and testing conditions?

A1: We would like to clarify that BC-only systems rely solely on BC signals with limited high-frequency information and must therefore infer and reconstruct the missing mid- and high-frequency components, which may pose challenges for downstream speech recognition. In contrast, AC–BC systems can use the richer mid- and high-frequency information in the AC signal to provide more direct acoustic evidence for reconstruction, helping preserve mid- and high-frequency details consistent with the original speech. To further compare BC-only and AC–BC systems, we have added EBEN [1] as a BC-only baseline. The table below reports the average performance of EBEN and FocusMe on the test set:

ModelModalityParams. (M) ↓MACs (G) ↓SI-SDR (dB) ↑PESQ ↑STOI ↑ESTOI ↑DNSMOS ↑CER (%) ↓
EBENBC only1.950.75-0.171.340.690.542.8981.04
FocusMeAC + BC4.301.5011.272.480.880.813.1717.03

The results show that FocusMe outperforms EBEN, a BC-only system, in speech quality, speech intelligibility, and downstream speech recognition performance.

Q2: Why is there no comparison with MBENet? Could you clarify why MBENet was not selected as a baseline? Would it be possible to provide comparison results under the same experimental setup?

A2: We thank the reviewers for pointing this out. We initially selected baselines commonly used in prior studies of this task, including the DBMIF [2] study. MBENet [3] is also a relevant AC–BC baseline and should have been included. We have therefore added MBENet to the revised evaluation, retraining it under the same experimental settings and evaluating it on the same test set as the other models. The table below reports the average performance of MBENet and FocusMe on the test set:

ModelModalityParams. (M) ↓MACs (G) ↓SI-SDR (dB) ↑PESQ ↑STOI ↑ESTOI ↑DNSMOS ↑CER (%) ↓
MBENetAC + BC3.855.6712.602.220.850.773.1721.65
FocusMeAC + BC4.301.5011.272.480.880.813.1717.03

Compared with MBENet, FocusMe achieves a higher PESQ score, better speech intelligibility, and a lower CER. It also requires 73.5% fewer MACs (1.50 G versus 5.67 G), with 11.7% more parameters (4.30 M versus 3.85 M).

Q3: Were multiple random seeds used in the experiments? Could the authors clarify this and provide the mean and standard deviation of key metrics for FocusMe and the primary baseline methods, based on multiple independent training runs, to demonstrate the stability of the experimental results?

A3: The experiments reported in the original manuscript were not repeated across multiple random seeds. Following the reviewer’s suggestion, we have repeated the experiments using five randomly selected seeds to further assess the stability of the results. The table below reports the mean and standard deviation of each model’s performance on the test set across these five seeds:

ModelModalitySI-SDR (dB) ↑PESQ ↑STOI ↑ESTOI ↑DNSMOS ↑CER (%) ↓
EBENBC only0.34 ± 0.1591.33 ± 0.0090.70 ± 0.0020.54 ± 0.0052.91 ± 0.00379.69 ± 0.58
FCNAC + BC5.15 ± 0.2071.27 ± 0.0120.70 ± 0.0020.54 ± 0.0042.50 ± 0.00549.26 ± 0.56
DCCRNAC + BC10.25 ± 0.0322.37 ± 0.0070.87 ± 0.00060.80 ± 0.00083.14 ± 0.00520.63 ± 0.63
MMINetAC + BC11.63 ± 0.0771.79 ± 0.0100.84 ± 0.0010.74 ± 0.0032.83 ± 0.00926.85 ± 0.58
VibVoiceAC + BC8.81 ± 0.0341.98 ± 0.0060.82 ± 0.00030.74 ± 0.00062.97 ± 0.00533.28 ± 0.40
MBENetAC + BC12.98 ± 0.0302.26 ± 0.0270.86 ± 0.0030.79 ± 0.0043.20 ± 0.00922.76 ± 0.73
DBMIFAC + BC7.64 ± 0.3531.60 ± 0.0550.79 ± 0.0060.69 ± 0.0132.60 ± 0.08635.39 ± 0.95
FocusMeAC + BC11.66 ± 0.0182.52 ± 0.0080.89 ± 0.0010.82 ± 0.0023.19 ± 0.00516.41 ± 0.34

The results show that FocusMe retains the highest mean PESQ (2.52), STOI (0.89), and ESTOI (0.82), and the lowest mean CER (16.41%).

Q4: How does the method perform in real-world noisy environments? In practical applications, however, BC signals can be affected by factors such as how the device is worn, variations in contact, or physical movement; furthermore, AC and BC signals may not always be perfectly synchronized. Do the authors have experimental results for these scenarios, or have the authors tested the method using AC–BC recordings captured in actual, noisy environments?

A4: To evaluate performance under real-world noisy environments, we recorded paired AC and BC signals directly from a headset in a subway environment and compared FocusMe with the baseline methods on the same recorded segment. We have added this study to the appendix of the revised manuscript. Appendix E.1 of the revised manuscript describes the recording setup, and Appendix E.2 presents spectrograms of the speech outputs from the different models. The spectrogram comparison suggests that FocusMe suppresses noise more effectively than the baseline methods while preserving more speech detail. Although EBEN, a BC-only system, generates more mid- and high-frequency components, our informal listening suggested distortions in speaker timbre and possible changes in the semantic content of its output. Separately, we recorded speech under different wearing and movement conditions to examine sensitivity to practical changes in headset use. We evaluated six test conditions: (a) normal wearing, (b) head movement, (c) walking, (d) touching the headset by hand, (e) walking while only the interfering speaker was speaking, and (f) loose wearing. The table below summarizes the experimental results under conditions (a)–(d) (Table 9, Appendix E.3):

ConditionTarget only: WER (%) ↓Target only: ΔWER (pp)Target + interfering speech: WER (%) ↓Target + interfering speech: ΔWER (pp)
(a) Normal wearing1.070.004.300.00
(b) Head movement2.86+1.796.81+2.51
(c) Walking0.71−0.363.22−1.08
(d) Touching the headset2.15+1.084.65+0.35

The results indicate that FocusMe is robust to perturbations in the BC signal. Furthermore, FocusMe remains effective at suppressing interfering speech under condition (e), where the wearer is walking and only the interfering speaker is speaking (Figure 10, Appendix E.3). Loose wearing remains a failure case: FocusMe and all evaluated baselines perform poorly under this condition (Figure 11, Appendix E.3). Improving robustness to variable device contact is an important direction for our future work. Besides, we did not observe AC–BC temporal misalignment in the recordings collected with this headset.

Reference

[1] Julien Hauret, Thomas Joubaud, Veronique Zimpfer, and ´´Eric Bavu. EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones. In IEEE International Conference on Acoustics, Speech and Signal Processing, pp.1–5, 2023.

[2] Yilei Wu, Changyan Zheng, Xingyu Zhang, Yakun Zhang, Chengshi Zheng, Shuang Yang, Ye Yan, and Erwei Yin. DBMIF: A deep balanced multimodal iterative fusion framework for air- and bone-conduction speech enhancement. Applied Intelligence, 56(5):165, 2026.

[3] Chen Zhang, Linfeng Feng, Zhi Liu, Xiao-Lei Zhang, and Xuelong Li. MBENet: Bone-conduction and air-conduction fusion network for target speaker extraction. In IEEE Automatic Speech Recognition and Understanding Workshop, pp. 1–5, 2025.

Thank you for the additional experiments. I have three remaining questions.

  1. Why do EBEN’s results change across AC interference conditions? Appendix A states that all 13 conditions reuse the same 100 sets of source segments, and EBEN uses only BC. Yet Table 8 reports CERs of 73.44% at −10 dB SIR and 96.33% at −5 dB. Were the BC inputs, model checkpoint, preprocessing, and evaluation settings identical across conditions? If so, why do the results differ?

  2. Did anything besides the training seed change? The original manuscript says the baselines followed their original training settings, whereas the revision specifies a common optimizer, learning rate, and number of epochs. Was this a wording correction or a change in the experiments? Also, FocusMe’s SI-SDR changed from 11.27 to 11.66 ± 0.018. This gap is much larger than the reported standard deviation. Were the five runs trained independently from scratch, with all settings except the seed unchanged from the original experiment?

  3. How does FocusMe compare with the baselines? The paper mentions a reference text of about 300 words. For a sample of that length, one error changes WER by about 0.33 percentage points, so some differences in Table 9 may reflect only one or two errors. Since Table 9 shows only FocusMe, could you also report WERs for unprocessed AC, MBENet, and EBEN on the same recordings?

Q1: Why do EBEN’s results change across AC interference conditions? Were the BC inputs, model checkpoint, preprocessing, and evaluation settings identical across conditions? If so, why do the results differ?

A1: We thank the reviewer for pointing this out. We would like to clarify that Appendix A was intended to state that all baselines use the same speech samples within each test condition, whereas the selected samples may differ across conditions. Specifically, the AC–BC speech pairs for each condition are randomly sampled from a held-out test set with no overlap with the training data. Therefore, even though the EBEN checkpoint, preprocessing pipeline, and evaluation settings remain unchanged, its evaluation results may vary because different BC samples are used. The original wording was unclear and may have caused confusion, we will revise Appendix A to accurately describe the sampling procedure and clarify that the source data are consistent across baselines within each test condition.

Q2: Did anything besides the training seed change? The original manuscript says the baselines followed their original training settings, whereas the revision specifies a common optimizer, learning rate, and number of epochs. Was this a wording correction or a change in the experiments? Also, FocusMe’s SI-SDR changed from 11.27 to 11.66 ± 0.018. This gap is much larger than the reported standard deviation. Were the five runs trained independently from scratch, with all settings except the seed unchanged from the original experiment?

A2: We thank the reviewer for pointing this out. The original baseline experiments already used the common optimizer, learning rate, and number of training epochs specified in the revised manuscript. Our previous statement that we followed the original baseline settings referred specifically to the network configurations. The revision corrects the description of the training protocol; the training settings themselves remain unchanged. All five FocusMe runs were independently initialized and trained from scratch. The previously reported SI-SDR value of 11.27 dB came from a separate earlier run that was not included in the statistics for these five runs. The revised result of 11.66 ± 0.018 dB represents the mean and standard deviation across the five runs. We also observed similar performance improvements when retraining the other baselines with different random seeds. These changes may partly reflect training variability associated with different random seeds.

Q3: How does FocusMe compare with the baselines? Could you also report WERs for unprocessed AC, MBENet, and EBEN on the same recordings?

A3: We evaluated FocusMe, unprocessed AC, MBENet, and EBEN on the same recordings under four usage conditions. These experiments were conducted in an otherwise quiet environment under two speech settings: target speech only and concurrent target and interfering speech. The table below reports WERs, with the lowest value in each condition and speech setting shown in bold.

ConditionMethodTarget only: WER (%) ↓Target + interfering speech: WER (%) ↓
(a) Normal wearingUnprocessed AC0.3529.39
MBENet1.077.88
EBEN57.7086.73
FocusMe1.074.30
(b) Head movementUnprocessed AC2.1517.56
MBENet2.156.81
EBEN97.8481.72
FocusMe2.866.81
(c) WalkingUnprocessed AC2.869.31
MBENet4.305.73
EBEN92.4760.21
FocusMe0.713.22
(d) Touching the headsetUnprocessed AC2.514.65
MBENet3.587.52
EBEN74.5592.11
FocusMe2.154.65

With interfering speech, FocusMe achieves the lowest or joint-lowest WER across all four usage conditions. In the target-only setting, FocusMe achieves the lowest WER during walking and headset touching. Under normal wearing, it matches MBENet but has a higher WER than unprocessed AC (1.07% vs. 0.35%). FocusMe also achieves lower WER than EBEN in all eight combinations of usage condition and speech setting.

Reviewer_VtJPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to recover the wearer's speech from noisy audio with interfering speakers using synchronized bone-conduction signals. THe authors propose FocusMe, which combines AC–BC feature fusion with frame-level BC speaker embeddings to guide speech reconstruction.

Critical strengths
  • I am not familiar with AC-BC speech enhancement, but I feel that the proposed method makes a lot of sense (the techniques they used have been proposed and validated in other related (but different) fields. So from a technical point of view, I think their method may be able to advance the field of AC-BC speech enhancement.
Critical weaknesses
  • My main concern is that, since the paper mainly proposes a new architecture, I would expect stronger experimental evidence to establish its contribution at the level expected for ICLR. From my own reading (on paper content), the task seems closely related to TSE/PSE, with the main difference being the source and form of the conditioning input. Could existing TSE/PSE methods be adapted to this task? If so, comparisons with stronger baselines from these areas would help show the benefit of the proposed design. Even if the above answer is no, baselines are still limited and outdated, and some directly relevant works discussed in the paper are missing from the experiments. For example, why is MBENet not included?
  • My second concern is about the reliability of the experimental results. On the one hand, the parameter counts of these models vary a lot, so I wonder whether the comparison is fair. Besides, is there a standard benchmark for this task? I would like to see evaluation on such a benchmark, if available, and comparisons between architectures with similar parameter counts. On the other hand, some baselines seem to perform poorly. Were they reproduced and tuned properly? DCCRN is an old method, but some improvements over it seem small: STOI only increases from 0.86 to 0.88. Could simply adapting a more advanced SE backbone to the same AC–BC inputs achieve similar or better results? I would also like to see results across multiple training seeds and statistical significance tests.
  • The citation of DCCRN is incorrect (line 334).
Questions for authors

Please see weaknesses.

The current version is based on my reading of this paper. I am not familiar with this task and did not read any other paper in this field during the peer review process. I would change my score during the rebuttal phase (and I might raise new questions during the rebuttal phase according to authors' response). If I have misunderstood some part, I hope the author can clarify.

5 comments

Q1: The task seems closely related to TSE/PSE, with the main difference being the source and form of the conditioning input. Could existing TSE/PSE methods be adapted to this task?

A1: We thank the reviewers for their observations on the relevance of this task to TSE/PSE. We would like to clarify that TSE/PSE methods use a speaker encoder to extract target-speaker cues from separately recorded reference speech. In contrast, FocusMe obtains these cues directly from bone-conduction (BC) speech captured synchronously with the air-conduction (AC) signal, eliminating the need for additional prerecorded enrollment speech. Moreover, our causal frame-level speaker embedding (FSE) extractor leverages information in the synchronized BC signal that is closely related to the target AC speech, providing frame-level speaker cues with only 0.59M parameters. By comparison, ECAPA-TDNN [1], a widely used speaker encoder in TSE/PSE methods, contains 22.3M parameters.

Q2: Baselines are still limited and outdated, and some directly relevant works discussed in the paper are missing from the experiments. For example, why is MBENet not included?

A2: First, we would like to clarify that the original baselines are widely used for this task and were also adopted in the DBMIF [2] study. Our evaluation includes DBMIF itself and has been expanded to include EBEN [3], a BC-only system, and MBENet [4], the method recommended by the reviewer. All baselines were retrained and evaluated under the same experimental settings. The following table reports the weighted averages over the all test condition:

ModelModalitySI-SDR (dB) ↑PESQ ↑STOI ↑ESTOI ↑DNSMOS ↑CER (%) ↓
EBENBC only-0.171.340.690.542.8981.04
MBENetAC + BC12.602.220.850.773.1721.65
FocusMeAC + BC11.272.480.880.813.1717.03

FocusMe outperforms EBEN and MBENet in terms of speech quality (PESQ), speech intelligibility, and downstream speech recognition performance. MBENet achieves higher SI-SDR (12.60 versus 11.27 dB). We have added the results of the supplementary experiments to the revised manuscript (Tables 2 & 3, Sec. 5.1).

Q3: The parameter counts of these models vary a lot, so I wonder whether the comparison is fair. Besides, is there a standard benchmark for this task? Some baselines seem to perform poorly. Were they reproduced and tuned properly?

A3: As discussed in A2, these systems are commonly used as baselines for this task. We further compare parameter counts and computational costs to demonstrate FocusMe’s greater overall efficiency. There is currently no standardized evaluation benchmark for this task, and closely related studies use different protocols to construct test mixtures. For example, DCCRN [5] uses mixtures constructed from ESMB recordings at SNRs of −5, 0, and 5 dB, whereas DBMIF constructs mixtures from ABCS recordings with different noise sources at SNRs ranging from −15 to 5 dB. To ensure as fair a comparison as possible, we retrained all baselines under the same experimental settings and evaluated them on the same test set, without otherwise modifying the baseline systems.

Q4: The citation of DCCRN is incorrect (line 334). DCCRN is an old method, but some improvements over it seem small: STOI only increases from 0.86 to 0.88. Could simply adapting a more advanced SE backbone to the same AC–BC inputs achieve similar or better results?

A4: We would like to clarify that DCCRN is the AC–BC sensor-fusion system, which uses a densely connected CRN with noisy AC and synchronized BC inputs. It is distinct from the single channel speech enhancement system [6]. Although “DCCRN” is a commonly used name in air- and bone-conduction speech enhancement, we have added a table (Table 1, Sec. 5.1) to the revised manuscript summarizing the details of each model. We also note that FocusMe achieves an average STOI gain of 0.02 over DCCRN, which is modest in absolute terms. Complementary metrics show that PESQ increases from 2.29 to 2.48, while CER decreases from 22.85% to 17.03%, corresponding to a reduction of 5.82 percentage points (approximately 25.5% relative). These results indicate meaningful improvements in speech quality and downstream speech recognition performance.

Q5: I would also like to see results across multiple training seeds and statistical significance tests.

A5: Following the reviewer’s suggestion, we repeated the experiments using five randomly selected seeds (the seeds are 42, 123, 3407, 2456, 5593) to assess the stability of the results. The table below reports the mean and standard deviation of each model’s performance on the test set across these five seeds:

ModelModalitySI-SDR (dB) ↑PESQ ↑STOI ↑ESTOI ↑DNSMOS ↑CER (%) ↓
EBENBC only0.34 ± 0.1591.33 ± 0.0090.70 ± 0.0020.54 ± 0.0052.91 ± 0.00379.69 ± 0.58
FCNAC + BC5.15 ± 0.2071.27 ± 0.0120.70 ± 0.0020.54 ± 0.0042.50 ± 0.00549.26 ± 0.56
DCCRNAC + BC10.25 ± 0.0322.37 ± 0.0070.87 ± 0.00060.80 ± 0.00083.14 ± 0.00520.63 ± 0.63
MMINetAC + BC11.63 ± 0.0771.79 ± 0.0100.84 ± 0.0010.74 ± 0.0032.83 ± 0.00926.85 ± 0.58
VibVoiceAC + BC8.81 ± 0.0341.98 ± 0.0060.82 ± 0.00030.74 ± 0.00062.97 ± 0.00533.28 ± 0.40
MBENetAC + BC12.98 ± 0.0302.26 ± 0.0270.86 ± 0.0030.79 ± 0.0043.20 ± 0.00922.76 ± 0.73
DBMIFAC + BC7.64 ± 0.3531.60 ± 0.0550.79 ± 0.0060.69 ± 0.0132.60 ± 0.08635.39 ± 0.95
FocusMeAC + BC11.66 ± 0.0182.52 ± 0.0080.89 ± 0.0010.82 ± 0.0023.19 ± 0.00516.41 ± 0.34

The results show that FocusMe retains the highest mean PESQ (2.52), STOI (0.89), and ESTOI (0.82), and the lowest mean CER (16.41%).

Reference

[1] Desplanques B , Thienpondt J , Demuynck K .ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification[J]. 2020.DOI:10.21437/Interspeech.2020-2650.

[2] Yilei Wu, Changyan Zheng, Xingyu Zhang, Yakun Zhang, Chengshi Zheng, Shuang Yang, Ye Yan, and Erwei Yin. DBMIF: A deep balanced multimodal iterative fusion framework for air- and bone-conduction speech enhancement. Applied Intelligence, 56(5):165, 2026.

[3] Julien Hauret, Thomas Joubaud, Veronique Zimpfer, and ´´Eric Bavu. EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones. In IEEE International Conference on Acoustics, Speech and Signal Processing, pp.1–5, 2023.

[4] Chen Zhang, Linfeng Feng, Zhi Liu, Xiao-Lei Zhang, and Xuelong Li. MBENet: Bone-conduction and air-conduction fusion network for target speaker extraction. In IEEE Automatic Speech Recognition and Understanding Workshop, pp. 1–5, 2025.

[5] Heming Wang, Xueliang Zhang, and DeLiang Wang. Fusing bone-conduction and air-conduction sensors for complex-domain speech enhancement. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:3134–3143, 2022a.

[6] Hu Y , Liu Y , Lv S ,et al.DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement[J]. 2020.DOI:10.21437/Interspeech.2020-2537.

I thank the authors for providing additional results. After careful consideration, I have decided to maintain my score. The reasons why I am not advocating for this paper to be accepted at ICLR are:

(1) This is an architecture-design paper. The performance improves on the authors' own benchmark, but not by much. Considering that the model is larger than some baselines, it remains unclear whether their design is truly better. I also can't be sure whether industry will use such a model in practice. I would be more willing to strongly advocate for papers that use fewer resources but achieve better results, or use more resources but achieve significant improvements over existing works (like recent large models that show scaling phenomena).

(2) Although I am not very familiar with this specific field, I am not surprised by the architectural details they designed. From a broader speech enhancement perspective, I feel that I have already seen similar designs before. Based on my personal reading, I just feel that this work does not meet the standards of ICLR. I think papers at ICLR, as a top-tier conference, should provide more inspiration for researchers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.