acceptodds
Under review as a conference paper at ICLR 2027

Situated Inference: Adapting Frozen Generative Speech Enhancement under Acoustic Distribution Shift

Abstract

Diffusion and flow matching models typically enhance speech using fixed inference strategies that may become unsuitable under acoustic distribution shifts in real-world deployment. Our cross-condition analysis shows that the advantage of an inference strategy is not consistently preserved across acoustic conditions, even when the model weights and solver update rule remain fixed. This finding motivates Situated Inference, which specializes the inference strategy of a frozen generative enhancer to its situation, the target acoustic distribution under which it is deployed. Using paired target speech, Situated Inference evaluates a structured inference space that jointly varies observation processing, the reverse starting point, and the solver step count, thereby constructing a target-specific quality and computation profile. From this profile, it selects one strategy either to maximize enhancement quality or to minimize computation subject to the default quality, then fixes that strategy for subsequent inputs. Across four generative backbones and three acoustic conditions, Situated Inference improves perceptual speech quality under quality specialization and reduces the number of function evaluations under efficiency specialization. Deployment on the Ameca humanoid further illustrates the efficiency advantage of Situated Inference in practice, with demonstrations available at https://anonymous.4open.science/w/situated-inference-demo-0761/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Generative Speech Enhancement, Diffusion Models, Humanoid Audition

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8kwoOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies target-condition-specific configuration selection for frozen generative speech enhancement models. For each pretrained backbone, it evaluates a finite set of configurations comprising observation preprocessing, the reverse-process starting point, and the number of solver steps. Using paired noisy/clean speech from the target condition, it selects either the configuration with the highest average PESQ or the lowest-NFE configuration whose estimated PESQ matches or exceeds the default. The selected configuration is then fixed for subsequent inputs. Experiments cover four pretrained backbones and three acoustic conditions, with additional analyses of configuration transfer, component contributions, target-set size, and an Ameca robot demonstration.

Critical strengths

The evaluation includes four generative backbones and three acoustic conditions, with configuration selection separated from held-out evaluation. This provides evidence that the observed benefits of target-specific configuration selection are not confined to one backbone or to the samples used for selection. The component ablations also help identify which configuration choices contribute to the gains. These are useful experimental checks.

Critical weaknesses
  1. Limited contribution beyond standard configuration tuning. The method searches over preprocessing, reverse starting point, and step count for existing frozen models, then selects a configuration using paired target data. The paper does not establish a substantial methodological contribution or sufficiently informative empirical findings beyond conventional validation-based configuration selection.

  2. Limited applicability due to the requirement for paired target data. In general real-world deployment, only noisy recordings are available, without corresponding clean references. The method nevertheless requires representative noisy/clean pairs from the target environment, making its configuration-selection procedure inapplicable when such supervision is unavailable. Even when these pairs can be obtained, the paper does not establish an advantage over independent adaptation methods using the same supervision: its main external baselines do not receive paired target data.

Questions for authors
  1. What does the proposed method contribute beyond standard validation-based configuration search?

  2. How exactly were the Ameca noisy/clean pairs acquired and aligned, and how many were used?

  3. How does the method compare with alternative adaptation approaches using the same paired target data?

Reviewer_sxcMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Generative speech enhancement models typically use fixed inference strategies, which may become unsuitable when acoustic conditions change. The authors propose Situated Inference, which jointly selects observation processing, the reverse starting point, and the solver step count while keeping the model weights and solver update rule fixed. Using paired noisy and clean speech from the target acoustic condition, it selects a fixed strategy that maximizes estimated quality or minimizes the number of function evaluations (NFE) while preserving estimated default quality. Across four generative backbones and three acoustic conditions, maximizing quality improves held-out PESQ, while minimizing NFE and maintains or improves PESQ.

Critical strengths
  1. The authors provide clear support for the motivation and design of Situated Inference. Figure 1 shows limited strategy transferability across acoustic conditions and the advantage of joint over step-only adaptation under the same NFE budget. These findings suggest that a backbone’s default inference configuration should not be assumed to remain suitable in a new acoustic environment, providing practical guidance for deploying generative SE models.

  2. The method achieves consistent benefits across four generative backbones by adjusting inference configurations while keeping model weights and the solver update rule fixed. The Quality objective improves held-out PESQ, while the Efficiency objective reduces NFE and maintains or improves PESQ. The method is straightforward to implement and avoids the cost of retraining the enhancement model.

Critical weaknesses
  1. Although the method improves PESQ, which is also used for strategy selection, ESTOI and SI-SDR do not consistently improve and sometimes decline in Table 1. The paper provides limited analysis of these differences, leaving the trade-offs between PESQ gains, intelligibility, and signal fidelity unclear.

  2. The only external adaptive baseline ArtiFree achieves lower PESQ than the default configuration in all 12 settings, but this consistent degradation is not explained. The paper also does not clearly specify how ArtiFree is implemented, including the SNR estimation method, category thresholds, and mapping to sampling depth. Providing these details would help confirm that the baseline is appropriately configured and make the comparison with existing adaptive inference more conclusive.

Questions for authors
  1. Could you report the selected strategy (m, tₛ, N) for each backbone, acoustic condition, and objective? For example, which observation processing and reverse starting point are selected for FlowSE under the Efficiency objective on LSM, where only one NFE is used?
  2. In some settings, PESQ improvements accompany decreases in ESTOI or SI-SDR. What quality trade-offs do these results reflect, and are they associated with changes to particular inference components?
  3. Could you explain why ArtiFree’s lower PESQ than the default configuration in all 12 settings?
Reviewer_vN3rOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work adapts the inference strategy of frozen diffusion/flow-based SE models to different acoustic conditions. The authors adjust three components (observation processing, the reverse starting point, and the solver step count), and use paired noisy and clean speech from each target scenario to select a strategy for better quality and efficiency. Unlike instance-level adaptation, the selected strategy is reused for later inputs from the same condition, so the search cost can be shared across these inputs.

Critical strengths

The merits of this work are two-fold. First, it has certain value: for SE systems deployed in specific and fixed scenarios, this scenario-aware inference strategy is cost-manageable, and can improve the performance of the SE system. Second, from the technique perspective, the method makes sense to me. It follows a similar idea to noise-aware SE, where the system adapts its processing to the acoustic condition. From this perspective, adapting the inference strategy to the target environment seems reasonable.

Critical weaknesses

My concern is that this method lacks theoretical guidance, so all hyperparameters choices are based on experience. However, selecting these hyperparameters is expensive for diffusion/flow-based SE methods, because such methods need multiple steps to generate speech. Plus, the effectiveness of this method also depends on the amount of available paired data. If data samples are limited, the estimated hyperparameters may be suboptimal (or even harmful -- although the results in the paper are good, I personally think the proposed method may harm performance).

I also have two suggestions.

Suggestion 1: Figure 1 appears too early in the paper, especially when I don’t know the method details yet. Without these details and with only a limited description of the experiments, it is hard for readers to understand Figure 1.

Suggestion 2: The ablation study does not include observation-only and start-only settings. If spectral preprocessing already improves PESQ, part of the gain may come from the front end rather than from adapting the generative sampling process. I suggest adding these two settings, as well as a preprocessing-only baseline without the generative model, to clarify where the gains come from.

Questions for authors

Please see "Critical Weaknesses".

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.