acceptodds
Under review as a conference paper at ICLR 2027

Generative Sound Masking

Abstract

A common way to reduce the salience of disruptive background sounds is to perform sound masking: using a pleasant sound, like pink noise, to conceal the undesirable sounds. We propose a method that learns to mask future background sounds. First, we apply a novel diffusion guidance method to obtain sounds that successfully mask a given background sound, which we use as synthetic training data. Second, we use this data to train a causal diffusion model to generate mask sounds that successfully conceal future background noises. Our online model reduces background-identification accuracy in an audio-language model more than background-independent masking baselines. Additional automated metrics assess the natural-sound content and distributional similarity of the resulting mixtures.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
sound generation, sound masking

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xqy9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a generative sound masking method that generates synthetic background-mask training pairs using guided audio diffusion, filters them using automated LLM and audio-classifier checks, and trains a causal autoregressive diffusion model (distilled with Easy Consistency Tuning) to generate natural sounds that mask unwanted background noise.

Critical strengths

None

Critical weaknesses
  1. This paper is essentially a contrived alternative to standard audio source separation and noise suppression, and it fails to compare against proper state-of-the-art separation baselines such as TF-GridNet [A] and TF-Locoformer [B]. Without comparing against these models, the proposed setup reads like a gimmick with little practical value: the authors claim to predict a causal mask for future background sounds, but their online model is simply acting as a causal sequence predictor with lookahead, which a standard state-of-the-art causal separation model [A, B] can do directly and much more effectively. Furthermore, the authors normalize and decode the results of their so-called online causal algorithm in a non-causal way across the entire clip during evaluation.

  2. As shown in Section 3.4 and Section 5.3, the experimental setup is circular and lacks human evaluation: the authors filter and annotate their training data using LLMs and audio classifiers, and then evaluate their model on the exact same LLM and classifier labels without including any human listening study.

[A] Wang, Z.Q., Cornell, S., Choi, S., Lee, Y., Kim, B.Y. and Watanabe, S., 2023. TF-GridNet: Integrating full- and sub-band modeling for speech separation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31, pp. 3221-3236.

[B] Saijo, K., Wichern, G., Cornell, S., Germain, F.G., Khurana, S. and Le Roux, J., 2024, September. TF-Locoformer: Transformer with local modeling by convolution for speech separation and enhancement. In 2024 18th International Workshop on Acoustic Signal Enhancement (IWAENC) (pp. 205-209). IEEE.

Questions for authors

Why should one use a generative model to add synthetic noise on top of background noise instead of using a state-of-the-art causal source separation or enhancement model such as TF-GridNet [A] or TF-Locoformer [B], and how do they compare in a human listening study?

Reviewer_NkukOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes adding generated natural sounds to environmental recordings to make the original sound less identifiable. While the work is motivated by applications involving human listeners, its effectiveness is evaluated primarily through audio-language model recognition. The paper provides no subjective listening tests and does not employ established psychoacoustic masking models, leaving the relationship between its evaluation metric and actual perceptual masking unvalidated.

Critical strengths

The proposed problem is interesting and relatively novel. The paper's main contribution is a generative approach that demonstrates how adding naturalistic sounds can alter the way audio-language models classify sound mixtures, although this does not establish effective perceptual masking for human listeners.

Critical weaknesses

The paper builds on an unsupported relationship between suppression of sound-source recognition with perceptual sound masking and acoustic comfort. Generating an additional sound that obscures the identity of an unwanted source does not necessarily reduce its perceptual salience or annoyance and may instead increase overall disturbance. Without subjective listening experiments evaluating detectability, annoyance, and listening comfort, the paper does not establish that its method solves the motivating problem. This is particularly concerning because the proposed method explicitly adds acoustic energy to the environment, potentially replacing one disturbing soundscape with another.

The proposed spectral subtraction and magnitude-only waveform reconstruction can introduce spectral inconsistencies and reconstruction artifacts, particularly when the estimated mixture power falls below the background power. Although the authors acknowledge phase-dependent cross-terms and introduce an iterative correction, they provide no analysis of the resulting artifacts or their effect on the automated evaluation. Since training examples are selected using audio classifiers, it is unclear whether reduced recognition reflects genuine masking or classifier sensitivity to reconstruction artifacts. An artifact analysis and controlled perceptual evaluation are needed to disentangle these effects.

Finally, the real-time capability of the proposed system remains under evaluated; On page 6, Section 5.1, the authors state: Quantitative results for our model are obtained by predicting the full latent sequence and then decoding it in a single pass with the (non-causal) SoundCTM VAE. A causal model for online deployment is proposed but the paper does not report measured end-to-end streaming latency or establish whether causal decoding preserves the reported audio quality and masking performance.

Questions for authors

The training pairs are selected using pretrained audio classifiers and audio-language models, rather than established psychoacoustic criteria or human perceptual judgments. How do the authors establish that the model learns genuine auditory masking rather than simply producing sounds that alter the semantic predictions of these classifiers? In particular, what evidence demonstrates that the generated maskers reduce human detectability or intelligibility more effectively than conventional spectrally shaped or adaptively selected masking sounds?

Could the authors report measured end-to-end streaming latency and confirm whether the reported masking and audio-quality results are preserved when using per-frame causal VAE decoding rather than full-sequence decoding?

Reviewer_EYNXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes generative sound masking. They first apply diffusion guidance method to obtain sounds that successfully mask a given background sound, which serves as synthetic training data. Then they train a causal diffusion model to generate mask sounds that conceal future background noises.

Critical strengths
  1. The task is well defined. The mask must be made from past audio only, so it can be played live. This goes beyond picking a fixed sound. The data pipeline is also described with care.
  2. The authors are transparent about what their metrics do not show. They mention machine recognition may differ from human recognition, and that the distribution metrics do not measure comfort. These statements helped me see where the evidence stops.
Critical weaknesses
  1. All evidence about masking comes from machines, and there is no audio to hear. The motivation is about people: noise in a hotel, sleep, privacy, noise sensitivity. But there is no listening test. Masking is judged by Qwen3-Omni and CLAP. The paper itself says that lower machine recognition does not mean lower human recognition. It also says masks should be pleasant, yet nothing measures this pleasantness.
  2. The link in the abstract has no audio examples. So I cannot tell if the masks are pleasant or if the background is really harder for a person to notice.
  3. The gain over simple baselines is small, and the model is less natural than a random nature clip.
  4. Only one LLM judge is used, how can we be sure a single LLM as a judge is the correct choice?
  5. There seems to be some mismatch? The intro and Figure 1 use pink noise, but the experiments use white noise?
Questions for authors
  1. Have you run any listening test?
  2. See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.