acceptodds
Under review as a conference paper at ICLR 2027

Diffusion-Based Multi-Fidelity Surrogate Modeling via Noise-Level Supervision Allocation

Abstract

Learning-based surrogate models for physical fields offer an efficient alternative to repeated numerical simulations, but obtaining sufficient high-fidelity training data remains costly. Multi-fidelity modeling addresses this bottleneck by combining limited high-fidelity data with abundant low-fidelity data, often through correction mappings. Rather than learning a low-to-high correction, we propose a novel diffusion-based method that allocates multi-fidelity supervision across noise levels. At selected higher-noise stages, low-fidelity data substitute for part of the high-fidelity supervision. Our method is motivated by the observation that the discrepancy between noisy high- and low-fidelity distributions decreases as the diffusion noise level increases. For deterministic physical responses, we use the diffusion signal-to-noise ratio to quantify this distributional discrepancy and bound the resulting denoising-target bias. Multi-fidelity data are jointly used to train a single conditional diffusion model that predicts physical fields without low-fidelity inputs at inference. On two thermal-field benchmarks, our method reduces the prediction error by up to 90% relative to the single-fidelity model and achieves accuracy comparable to that of existing multi-fidelity models trained with 50 times as many high-fidelity samples.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Multi-fidelity modeling, Diffusion model, Surrogate model, Physical field prediction

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rW8DOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The trick here is to use cheap simulations at selected diffusion noise levels. High-fidelity fields supervise the entire denoising chain. Low-fidelity fields join only above a chosen noise boundary. I like the simplicity of that rule. The reported gains are large, but the comparisons leave their reliability unclear.

Critical strengths

The allocation rule is simple and easy to implement. Cheap data contribute directly during training, and inference needs no LF simulation. That practical setup appeals to me.

The scarce-data gains deserve attention. With five HF fields, simple-benchmark MAE falls from 1.1753 to 0.1173 K against single-fidelity diffusion. That's encouraging, even though the uncertainty isn't quantified.

Critical weaknesses

I'm not sure noise allocation deserves all the credit. Under Equation 10, changing kk also changes per-timestep LF weighting. I'd want uniform pooling and matched-backbone pretraining controls.

Both benchmarks are variants of the same thermal-layout task. One run per setting makes the data-efficiency evidence fragile. I'd want repeated subsets and a different PDE family.

This looks expensive for deterministic field prediction. Every output uses 2,000 reverse steps. I need measured runtime and total-cost comparisons against single-pass surrogates.

Questions for authors

Do the gains survive matched training budgets? Please give the fine-tuning baselines comparable HF exposure and hyperparameter searches. That's the comparison that would most affect my score.

How was kk selected? Did the 200 paired HF fields in Figure 3 influence tuning? I'd count those labels toward the budget if they did.

What happens with k=1k=1, LF-only training, and matched-backbone pretraining? I'd also hold per-timestep LF weights fixed during the boundary sweep. That would isolate the allocation effect more cleanly.

How does noise-target contraction translate into final-field accuracy? The direct reconstruction rescaling cancels the SNR factor. I'd want a reverse-process error argument that addresses this gap.

Reviewer_5uFcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Noise-Level Supervision Allocation (NLSA), a diffusion-based multi-fidelity surrogate modeling framework for physical-field prediction. The main idea is to use high-fidelity (HF) data throughout the diffusion training process while incorporating low-fidelity (LF) data only at higher noise levels. The motivation is that Gaussian corruption progressively reduces the discrepancy between LF and HF distributions, allowing LF observations to provide useful denoising supervision with reduced fidelity-induced bias.

The authors derive an exact relationship between the conditional KL divergence and the diffusion signal-to-noise ratio for deterministic physical responses, together with a bound on the denoising-target bias arising from mixed-fidelity supervision. The resulting method trains a single conditional diffusion model using HF and LF denoising losses, without requiring LF simulation outputs at inference.

The method is evaluated on two steady-state temperature-field prediction benchmarks with different HF data budgets. The reported results show substantial improvements over single-fidelity diffusion, particularly when only a few HF samples are available, and competitive performance relative to two existing multi-fidelity surrogate baselines. The authors also examine the effect of the noise-level supervision boundary.

Critical strengths
  1. Simple and effective use of multi-fidelity supervision. The proposed method is straightforward to implement and requires only one shared conditional diffusion network, without introducing an explicit LF-to-HF correction network or requiring LF simulations at inference. The idea of allocating supervision according to the diffusion noise level is intuitive and provides a useful alternative perspective on how multi-fidelity data can be combined.

  2. Promising performance under extremely limited HF supervision. The reported improvements are substantial, particularly with only 1–10 HF training samples. The method also demonstrates advantages over the two multi-fidelity baselines in the smallest HF-budget regimes. The supervision-boundary ablation further suggests that the selection of noise levels for LF supervision meaningfully affects predictive performance. These findings indicate potentially significant practical value for data-scarce physical surrogate modeling.

Critical weaknesses

Limited methodological novelty and close algorithmic connection to Ambient Diffusion Omni.

The central idea of NLSA is closely related to Ambient Diffusion Omni: Training Good Models with Bad Data (Daras et al., NeurIPS 2025). The connection goes beyond the general observation that Gaussian noise reduces distributional discrepancies; the two methods share essentially the same high-noise supervision-allocation strategy.

First, the training procedures have nearly identical supervision-allocation structures. Ambient Omni uses high-quality observations throughout diffusion training while restricting lower-quality observations to noise levels above a prescribed threshold. NLSA adopts the same structure. The corresponding supervision-allocation rules can be expressed as

ALOmni(σ,z)=1[σ≥σmin⁡(z)],ALNLSA(t)=1[t≥k]A_L^{\mathrm{Omni}}(\sigma,z)=\mathbf{1}[\sigma\geq\sigma_{\min}(z)],\qquad A_L^{\mathrm{NLSA}}(t)=\mathbf{1}[t\geq k]

Here, the first expression abstracts the high-noise eligibility rule of Ambient Omni, rather than reproducing its complete training objective. In both methods, lower-quality observations contribute to a shared denoising network through noise-restricted training losses, without an explicit low-to-high correction network or inference-time guidance. NLSA additionally conditions the denoiser on the physical input xx and uses a different denoising objective. Nevertheless, the underlying supervision-allocation mechanism is substantially the same.

Second, the supervision-boundary selection is also closely related. Ambient Omni considers a globally selected minimum usable noise level and further proposes a time-dependent classifier to determine when high- and low-quality distributions become sufficiently difficult to distinguish, allowing sample-dependent thresholds σmin⁡(z)\sigma_{\min}(z). NLSA instead adopts a single fixed boundary kk, selected empirically through an ablation study. Thus, NLSA's allocation rule can be viewed as a simpler, fixed-threshold specialization of the high-noise supervision strategy already explored in Ambient Omni, rather than a fundamentally new allocation mechanism.

Third, the theoretical motivations are directly connected. Ambient Omni establishes that Gaussian smoothing contracts distributional discrepancies, thereby motivating the use of lower-quality observations at higher noise levels. NLSA specializes this principle to deterministic conditional HF/LF responses. Specifically, rescaling the forward diffusion variable gives

zt=ytαˉt=Ff(x)+σtϵ,σt2=1−αˉtαˉt=1SNRt.z_t=\frac{y_t}{\sqrt{\bar\alpha_t}}=F_f(x)+\sigma_t\epsilon,\qquad \sigma_t^2=\frac{1-\bar\alpha_t}{\bar\alpha_t}=\frac{1}{\mathrm{SNR}_t}.

Under this additive Gaussian parameterization, NLSA's Proposition 1 becomes

DKL(qtH∥qtL)=∥δ(x)∥22σt2=12SNRt∥δ(x)∥2,D_{\mathrm{KL}}(q_t^H\|q_t^L)=\frac{\|\delta(x)\|^2}{2\sigma_t^2}=\frac{1}{2}\mathrm{SNR}_t\|\delta(x)\|^2,

where δ(x)=FH(x)−FL(x)\delta(x)=F_H(x)-F_L(x). This is the standard equal-covariance Gaussian KL identity, providing an exact characterization for the deterministic conditional setting. The resulting contraction is consistent with the more general Gaussian-smoothing contraction principle analyzed in Ambient Omni.

Furthermore, NLSA's denoising-target bias bound follows from the fact that the optimal mixed-fidelity denoiser is a posterior-weighted combination of the fidelity-specific optimal denoisers. While this result is mathematically valid, it follows directly from the conditional-mean solution under squared loss. More importantly, Ambient Omni explicitly analyzes the bias-variance tradeoff associated with incorporating lower-quality observations, whereas NLSA's analysis characterizes only fidelity-induced bias. It does not establish when mixed-fidelity supervision improves HF prediction error or provide a principled rule for selecting kk.

I acknowledge that NLSA addresses conditional physical surrogate modeling rather than natural-image generation, and that its ordinary DDPM noise-prediction objective differs from Ambient Omni's degradation-aware training objective. The application-specific formulation may be practically useful, particularly because LF simulations are not required at inference. Nevertheless, the main algorithmic ingredients—noise-dependent eligibility of lower-quality observations, shared-denoiser training, and threshold-based supervision allocation—have already been substantially developed in Ambient Omni. It remains unclear what new methodological contribution arises specifically from the multi-fidelity physical setting beyond applying this established principle with a fixed threshold.

Questions for authors

Q1. Methodological distinction from Ambient Diffusion Omni. Ambient Diffusion Omni (NeurIPS 2025) already employs high-quality supervision across all noise levels and lower-quality supervision only above selected noise thresholds, including globally fixed and sample-dependent thresholds. This appears closely related to NLSA's training objective. Could the authors clarify the specific algorithmic novelty beyond adapting this supervision-allocation principle to conditional multi-fidelity surrogate modeling? In particular, which aspect of NLSA cannot be recovered by applying Ambient Omni's high-noise allocation strategy to conditional physical-field prediction?

Q2. Isolating the contribution of noise-level allocation. The reported improvements are substantial, but it is unclear how much comes from the proposed allocation mechanism rather than simply incorporating abundant LF observations through joint training. Could the authors provide a comparison with joint HF/LF diffusion training without noise-level restriction (i.e., (k=1)), using the same backbone and training configuration? A representative experiment in the scarce-HF regime would be sufficient for the rebuttal. If additional training is infeasible, please clarify whether existing results support a distinction between these two mechanisms.

Q3. Intrinsic LF-HF discrepancy and robustness. Since both fidelity levels solve the same thermal problem using different grid resolutions, how accurate are the interpolated LF simulation outputs relative to the HF reference fields? Reporting this error on the existing paired simulation data would help establish the difficulty of the multi-fidelity prediction problem. Additionally, the substantial nonmonotonic variation across HF budgets raises concerns about sensitivity to the selected HF subsets. Could the authors clarify how these subsets were selected and whether any repeated-run results or variability estimates are available?

Reviewer_2hwnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Noise-Level Supervision Allocation (NLSA), a diffusion-based multi-fidelity surrogate modeling method that uses low-fidelity data as supervision only at higher diffusion noise levels. This approach aims to reduce the amount of costly high-fidelity simulation data needed to train accurate surrogates of physical fields. The experimental results on two thermal-field benchmarks show that NLSA substantially reduces prediction error compared with the single-fidelity model and matches existing multi-fidelity methods while using fewer high-fidelity samples.

Critical strengths

The paper is well-written and easy to follow.

The core idea of applying noise-level supervision allocation to multi-fidelity surrogate modeling is intuitive and allows no low-fidelity input at inference time.

The experimental results seem strong, with the large gains (90% error reduction) especially when high fidelity data is scarce.

Critical weaknesses

The major concern is that the core idea of using lower-quality data only at high noise levels, with a bound showing the discrepancy shrinking with noise, was previously proposed in Ambient Diffusion Omni [1], which is not cited. This makes the methodological and theoretical novelty unclear beyond the application to multi-fidelity surrogates.

The experimental evaluation is limited to two thermal-field benchmarks from a single source. The claim that the principle is general across physical systems is therefore untested. It would be good to include more multi-fidelity PDE tasks such as the Navier-Stokes equation for fluid simulation.

The inference cost difference is significant and lacking discussion. Every NLSA prediction needs 2,000 denoising steps, while the DMFM and PD-DMFM baselines are single-pass deterministic networks. For a surrogate meant to replace simulation, the speedup comparison against baselines over the actual solver should be discussed.


[1] Daras, Giannis, et al. "Ambient diffusion omni: Training good models with bad data." Advances in Neural Information Processing Systems 38 (2026): 38064-38101.

Questions for authors

Please refer to the strengths and weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.