acceptodds
Withdrawn from ICLR 2027

Sensitivity Decomposition for Predictor-Conditioned Diffusion Speech Enhancement

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion-based speech enhancement systems often combine a deterministic predictor with a generative refinement stage, but it is not clear which component determines how the system responds when test conditions change. We study this question in StoRM, where the predictor output both initializes the reverse diffusion process and acts as its mean-reversion target, while the original noisy mixture is supplied separately to the generative stage. We show that the sensitivity of the enhanced output to a change in noise level can be decomposed exactly into contributions associated with these two conditioning routes. This decomposition reveals a structural asymmetry such that perturbations entering through the predictor affect the reverse process from its initial state, whereas perturbations entering only through score conditioning act through the subsequent dynamics. Our measurements show that the diffusion stage does not simply copy the predictor output; it substantially reshapes perturbations introduced through this route. Nevertheless, the predictor-associated contribution remains strongly aligned with the overall response of the enhanced signal. Extending the analysis from local sensitivities to finite changes in noise level leads to the same conclusion: near the training condition, the predictor route determines the direction of the output change, while the direct noisy-mixture conditioning contributes a substantially smaller, largely complementary component. Replacing the predictor-based reference with the noisy mixture causes the system to track the noisy signal instead of enhancing it, providing an independent intervention on the same mechanism. Together, these results identify the conditioning route that anchors the reverse process as the dominant source of robustness behavior in the model studied here. The analysis extends naturally to other differentiable generative samplers with multiple conditioning inputs.

Primary area:
generative models
Keywords:
diffusion models, speech enhancement, conditional generative models, sensitivity analysis, model attribution, robustness, score-based generative modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xTnROpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies a very specific question in a particular predictor-conditioned diffusion-based speech enhancement method, StoRM: how changes in noise level affect the enhanced output through its two conditioning routes, which are the predictor estimate and the original noisy mixture. Holding the sampler’s randomness fixed, the authors derive an exact decomposition of output sensitivity into contributions from these routes. They extend this decomposition to finite changes in noise level by integrating along the perturbation path and measure the contributions using automatic differentiation. Experiments on VoiceBank/DEMAND show that, in the evaluated settings, the predictor-associated contribution dominates the output response, although diffusion substantially transforms the perturbations entering through this route. A separately trained control anchored on the noisy mixture provides little enhancement, which the authors interpret as additional evidence for the importance of the anchoring signal.

Critical strengths
  1. Mathematical soundness. The sensitivity decomposition appears mathematically sound under the stated assumptions, and its extension from local sensitivities to finite output changes is consistent.
Critical weaknesses
  1. Limited scope and research insight. The empirical study focuses on a single architecture and dataset under noise-amplitude changes. The main finding provides limited guidance for methodological improvements, and no improvement motivated by the analysis is demonstrated. Moreover, although the decomposition is mathematically exact, predictor-route dominance is observed in configurations where diffusion provides limited gains over the predictor alone. It therefore remains unclear whether this finding reflects a meaningful architectural property or mainly the limited effectiveness of the refinement stage.
  2. The presentation obscures the motivation and interpretation. It is not easy to understand the motivation of this research. Readers need a clearer explanation of why separating these two routes matters, how the influence of each route differs from the contribution of each processing stage, and what the resulting attribution reveals about robustness. The lack of clarity on these central questions makes the paper difficult to follow and its contribution difficult to assess.
Questions for authors

Would predictor-route dominance persist in configurations where diffusion substantially improves upon the predictor alone? Could the authors provide evidence distinguishing an architectural effect from a consequence of limited generative refinement?

Reviewer_DKpqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks which conditioning route determines how StoRM, a predictor-conditioned diffusion speech enhancer, responds to changes in test-time noise level. The predictor output sets the sampler's initial state and drift target and also conditions the score network; the noisy mixture only conditions the score network. Fixing the sampler noise, the authors differentiate the executed sampler, split the output sensitivity exactly into an anchored and a modulating term, and integrate both along the perturbation path. On VoiceBank/DEMAND, the anchored perturbation is strongly transformed yet stays aligned with the total response, and a noisy-anchored control barely enhances. The paper concludes that the anchoring route dominates robustness in this model.

Critical strengths
  1. A useful question, made measurable: Robustness curves cannot tell whether an enhancer passes the predictor's errors through or transforms them. Differentiating the executed sampler separates the two routes for a trained checkpoint without retraining. For the measured checkpoints, with one input perturbed at a time, the anchored perturbation is far from passed through yet aligned with the total response (Table 1), which StoRM's own ablations do not show. This diagnostic is the paper's main contribution.

  2. Careful, checkable measurement: The derivatives are exact for the sampler actually run, the two terms add up to an independently computed total, and the path integration is justified and checked for convergence. My concerns are therefore about interpretation, not correctness.

Critical weaknesses
  1. The route split does not identify the claimed anchoring mechanism. The abstract calls the anchoring route "the dominant source of robustness behavior in the model studied here". However, the split between the two terms is not determined by the system's response to the noise change. The score network is trained, and queried by the sampler, only on inputs whose anchor equals the predictor output, and Section 7 notes that the split partly reflects extrapolation. In fact, adding to the score any smooth term that vanishes on all such inputs leaves every output and the total sensitivity unchanged, yet can shift equal and opposite amounts between the two terms. Within smooth functions, a suitable choice can even make the anchored term an exact copy of the predictor perturbation in sampler coordinates, although a network of the same width need not realize this. The fixed-checkpoint measurements stand, but the mechanistic reading does not follow. Together with the control issue below, this is my main concern. If the frozen-anchor term (question 1) carries the reported dominance, that issue is addressed and questions 2–3 are answered, I would consider weak acceptance. Otherwise, the claims should be stated as route attribution for this network.

  2. The only intervention removes the predictor, not just the anchor. The abstract calls the noisy re-anchoring "an independent intervention on the same mechanism". However, Appendix A11 defines it as a separately trained network conditioned on and initialized from the noisy mixture, with no predictor. It thus also removes the predictor's conditioning and changes the learning problem, so its failure to enhance cannot isolate anchor placement; the authors rightly confine that result to this 32-channel control (Section 7). Moreover, the anchored term includes the generator's transformation of the predictor perturbation. Route dominance, even at both widths, therefore does not show what Section 1 suggests: that generator-only changes cannot improve robustness. Contribution (iii) and the design conclusions rest on these inferences. I suggest describing the control as predictor removal and narrowing those conclusions. A placement claim would need a control that keeps the predictor but moves the anchor.

Questions for authors
  1. Could the authors report the frozen-anchor term, locally and over the finite-change window? This is the DD-only JVP of Eq. 21 with yy fixed and the score network's DD input held constant, so that the perturbation moves only the initial state and drift target. It is unaffected by the added score term in weakness 1 but still depends on the trained network. It needs one sampler JVP per integration node, without retraining.
  2. In which representation are the norms and cosines computed: the compressed STFT domain of Appendix A4, or the waveform described in Sections 1 and 6.3? Nonlinear decoding does not preserve angles.
  3. Which predictor does StoRM-128 use, and how much does its generative stage add to it? At 32 channels it adds little (Appendix A2), so the answer shows whether the attribution holds where the generator does real work.
  4. The discussion suggests a design lesson for predictor-conditioned enhancers in general, yet notes that another architecture could reverse the attribution. Should the lesson be limited to StoRM, or is there evidence beyond it?
Reviewer_u7pWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how increased test-time noise affects the output of StoRM, which first uses a predictor to denoise speech and then refines it with diffusion. It decomposes the output change into contributions from the predictor-output path and the noisy-input path, and finds that the predictor-output path dominates under the tested conditions.

Critical strengths

I like the motivation behind this paper (analyzing model behavior). The approach to the research questions is quite unique for me and I've learned some valuable things from reading the paper.

Critical weaknesses
  • In StoRM, the predictor output DD appears explicitly in the OUVE drift, but this design is not shared by all diffusion-based models. So I think its impact on the community is very limited.
  • The paper is very difficult to follow and relies heavily on prior knowledge of StoRM. For example, Section 3.2 refers to Assumption 1, but the assumption is only stated on page 18. Some symbols are also not explained when they first appear, such as cc in Eq. (5). The notation can be confusing: X0X_0 denotes the final output in Section 3.1 but the initial sampling state in Section 3.2, without a clear explanation of this change. More explanations of the notation and the intuition behind the equations would make the paper much easier to understand (I must say, without AI, I wouldn't be able to understand this paper).
  • The sensitivity analysis makes sense, but I am still unsure what it tells us about speech enhancement quality. A path that causes larger output changes is not necessarily more useful, and a less sensitive path may still provide important information.
  • The analysis provides some insight into how StoRM behaves, but it is still unclear how to use these findings to improve the model.
Questions for authors

I may not have fully understood some parts of the paper, and I would appreciate the authors' feedback during the rebuttal. Please clarify if I have misunderstood anything, thanks!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.