acceptodds
Withdrawn from ICLR 2027

Diffusion Posterior Sampling with Uncertain Forward Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

25 views

Abstract

Diffusion posterior samplers reconstruct signals by combining a pretrained diffusion prior with a measurement likelihood specified by a forward model and a noise model. Calibration and reference data can characterize uncertainty in the forward model's parameters, such as blur kernels or detector responses. Fixing these parameters at estimated values leaves their uncertainty out of the likelihood, and parameter errors are thus absorbed into the reconstruction. We introduce an uncertainty-aware likelihood (UAL), a Gaussian approximation to the marginal likelihood obtained by integrating over the uncertain parameters. For each candidate signal, UAL propagates parameter uncertainty into a structured measurement covariance, reducing the quadratic penalty along residual directions that parameter variation can explain. Existing samplers can use UAL in their data updates while retaining their pretrained diffusion priors and sampling schedules. Under a fixed Gaussian nuisance model, we bound likelihood error from linearization and the resulting average divergence between the exact marginal and UAL posteriors. We compare samplers with and without UAL across six inverse problems spanning Gaussian and motion deblurring, CT, MRI, ultrasound, and fluorescence microscopy, and find improvements in reconstruction and ensemble prediction. UAL reduces geometric mean MSE by 9% to 20% across samplers in the main Gaussian-deblurring setting and by 51% to 92% in motion deblurring when non-blind samplers use the population-mean kernel. In motion deblurring, DAPS and PnP-DM with UAL also give 6.2 to 6.4 dB higher ensemble-mean PSNR than published BlindDPS and Fast Diffusion EM. On the ablation set, UAL outperforms trace-matched isotropic inflation for PnP-DM in both deblurring tasks; in motion deblurring, it also exceeds the best-tested fixed scalar data weighting by 3.3 dB for PnP-DM and 4.2 dB for DAPS in average PSNR.

Primary area:
generative models
Keywords:
diffusion models, inverse problems, posterior sampling, forward-model uncertainty, nuisance marginalization, computational imaging

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zqE3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a new likelihood term, Uncertainty-Aware Likelihood (UAL), that takes into account the uncertainty on the forward model. Based on the knowledge of the distribution of forward models, the new likelihood can be used in existing diffusion models restoration processes. The models relies on a linearization that induces an quantified error. UAL is run on many inverse problems and diffusion frameworks and outperforms standard diffusion models run with a likelihhod that do not take into account the uncertainty of the forward model.

Critical strengths
  • The UAL construction is original and provides a promishing model of the forward model uncertainties.
  • The experimental comparison is made on diverse inverse problems and diffusion samplers.
Critical weaknesses

Major :

  • The performance of oracle method (where the correct forward model is used to reconstruct) is not provided. The experimental results show that many diffusion models methods have unstable results with respect with the forward model. However, it is hard to conclude on the proposed method since the oracle performances are not provided.
  • The paper is not well written and hard to follow. Many important paragraphs (see details below) are unclear.
  • The proposed methods relies on a perfect knowledge of the distribution of forward models that is unknown in many situations.

Minor/typos:

  • Page 3 line 137-139: I recommend to make a separate paragraph for Langevin samplers since it is significantly different than diffusion priors.
  • page 3 line 160 : Which data-based are you refer to ?
  • page 3 line 156-161 : This paragraph should be re-written. The link between sentences is unclear.
  • page 4 line 201 : The notation JψJ_{\psi} is not well adapted since this quantity does not depend of ψ\psi. I suggest the notation J0J_0.
  • page 5 line 26 : sis_i is the singular value of what ?
  • page 5 line 233 : What conditions the differentiability of yˉ\bar y and SS induce on A\mathcal{A} or qq ?
  • page 5 line 236 : I suggest to write the derivative of lUALl_{UAL} for an easier following of the transition (5) to (6).
  • page 5 line 225-232 : This paragraph should be re-written, it is unclear and hard to follow. Please detail the computation and reasoning.
  • page 5 line 240-252 : This paragraph is unclear and hard to follow. Especially there is many new notations in a short paragraph and many references to the Appendices without detailed explanations. Please detail it to make it clear.
  • page 5 line 245 (same remark for page 14 line 752) : The second equation on ww is not part of the system, it is a consequence of the first equation, please re-write it.
  • page 7 line 347-351 : The sampler are not provided, please complete and make the table self-contained.
  • page 8 line 394 : Please do not use abbreviations without introduction such as "SE".
  • page 14 line 718-725 : No details are given in the proof, please provide the complete proof.
  • page 14 line 744 : I suggest to add in the beginning of the sentence "By Proposition 1 with the linearized forward model". Please recall the definition of SS and GG.
  • page 14 line 744-745 : What do you mean by the sentence "A single factorization [...] logdetGlog det G." ?
Questions for authors
  • Could we adopt this method for taking into account uncertainty in the prior knowledge ?
  • page 5 line 245 : Why this system Gared=UTΣe−1dG a_{red} = U^T \Sigma_e^{-1} d admits a solution ? This solution is unique ? Why this name areda_{red} ?
  • How to adapt the proposed method when the forward model distribution is unknown ?
  • Since the Langevin samplers such as PnP-ULA are proved to be stable to forward models (see page 3 line 137), what is the practical gain of UAL for these samplers ?
  • What is the additional computational cost of using UAL compare to the standard likelihood ?
Reviewer_teRsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose incorporating an uncertainty-aware likelihood (UAL) into diffusion posterior sampling methods for imaging inverse problems. This UAL approximates the distribution of uncertain parameters in the forward model (termed “nuisance” parameters) as a Gaussian with a mean and covariance derived from either a calibration set or a user-provided prescription. The authors incorporate the UAL into existing diffusion posterior samplers and test on several inverse problems, including Gaussian and motion deblurring, CT, MRI, ultrasound, and fluorescence microscopy. They report that incorporating the Gaussian UAL generally improves both image quality and probabilistic prediction upon the non-UAL version.

Critical strengths

None

Critical weaknesses

The proposed UAL is extremely marginal and insufficient for an ICLR publication. Approximating the forward-model distribution as a Gaussian and incorporating this into the measurement likelihood is straightforward and is an idea that has already been used in methods for blind inverse problems, such as expectation maximization (EM). A suggestion for improvement would be to better contextualize this work among existing methods for uncertain forward models, including EM and blind diffusion posterior sampling methods.

The method hasn’t been tested on realistic conditions. First of all, it was difficult to follow in Section 4 how exactly the Gaussian approximation of the nuisance parameters was derived. Was it prescribed to follow the same distribution that generated the test data, or was it calibrated on a dataset, and if it was calibrated, how were the calibration data obtained? The most realistic setting would be one where you have many measurements associated with slightly different nuisance parameters and you must determine how to characterize the uncertainty of the nuisance parameters. I’m not sure it’s realistic to assume that one already has a calibration dataset of true nuisance parameters.

I know that the prompt only asked for one or two weaknesses, but I would just like to note that the writing is quite dense and difficult to follow, with a lot of undefined technical jargon and acronyms, such as CRPS, variogram, DRUS, etc.

Questions for authors

None

Reviewer_KmxDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies diffusion-based image reconstruction with uncertain forward model parameters. The authors use a first-order Taylor expansion to account for parameter uncertainty in the measurement covariance. Based on this, they propose an uncertainty-aware likelihood (UAL) to modify the data consistency updates of existing diffusion-based inverse problem solvers. The paper also provides theoretical analysis and experiments across several imaging tasks.

Critical strengths

1.Meaningful problem formulation.

The paper considers how uncertainty in forward model parameters affects image reconstruction. By using a first-order Taylor expansion, the authors incorporate this uncertainty into a structured measurement covariance and construct a new likelihood function to improve existing diffusion-based inverse problem solvers. The motivation is clear, and the problem is practically relevant.

2.Extensive experiments and theoretical analysis.

The paper provides experiments across several inverse problems, along with theoretical analysis of likelihood approximation error and posterior approximation error. These results provide useful support for the proposed method.

Critical weaknesses

1.The presentation could be improved.

The abstract and introduction spend considerable space describing experimental improvements, while the main modeling assumptions and the scope of the proposed method receive less emphasis. I think these aspects should be presented more clearly to help readers better understand the method and its limitations.

2.The applicability of the forward model uncertainty assumptions needs further validation.

The proposed method assumes that measurement uncertainty can be described by a specified forward model and a distribution over its parameters. A first-order approximation is then used to construct the measurement likelihood.

While this assumption is reasonable, uncertainty in real-world blind inverse problems may not always be captured by parameter variations within a fixed forward model. For example, the forward model itself may be misspecified, or the parameter distribution at test time may differ significantly from the calibration distribution.

Although the authors evaluate some deviations in parameter statistics, the current experiments do not fully demonstrate how well UAL performs under these more general types of model mismatch.

Questions for authors

1.Could the authors further clarify how UAL differs from traditional blind inverse problem solvers, where the uncertainty may involve not only unknown parameters but also the structure of the forward model itself? In particular, does UAL still provide improvements when the true forward model falls outside the assumed model family or when the parameter distribution at test time differs from the calibration distribution?

2.Appendix Table 8 already provides an oracle comparison using the true blur kernel. However, in practical applications, we often have access to an estimated forward operator rather than the true one. Could the authors compare UAL against the same baseline solvers using per-image estimates of the forward operator instead of the population-mean operator? This would help demonstrate whether UAL still offers clear benefits when the baselines have access to more accurate, but imperfect, forward model estimates.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.