acceptodds
Under review as a conference paper at ICLR 2027

Towards Unbiased Diffusion Variational Inversion via Principled Posterior Matching

Abstract

Diffusion models provide expressive priors for imaging inverse problems, but scalable variational inference with an implicit, non-degenerate posterior remains challenging because the prior KL term depends on the score of the variational distribution. We propose Principled Posterior Matching (PPM), a score-based variational framework that supports both particle-based optimization for individual observations and amortized inference through a reconstruction network for rapid, single-pass reconstruction. For two distributions evolved under the same forward diffusion process, we use the KL-dissipation relation to express their finite-time discrepancy in terms of an integrated Fisher divergence and a terminal KL term. We further derive a pathwise gradient representation using a score-projection identity under explicit differentiability, integrability, and reparameterization assumptions. When the prior and variational scores are exact, Monte Carlo sampling yields an unbiased estimator of the resulting gradient. In practice, these scores are calculated by pretrained and auxiliary models, and its performance is therefore affected only by score-estimation and optimization errors. We evaluate PPM on a range of computational imaging problems, including natural-image inpainting, super-resolution, and deblurring, fluorescence-microscopy reconstruction, and synthetic radio-interferometric black-hole imaging. Across these tasks, PPM produces more diverse reconstruction samples while maintaining competitive reconstruction fidelity relative to existing approaches. These results demonstrate that PPM provides a unified approach to sample-based variational inference and efficient amortized reconstruction across diverse imaging inverse problems.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computational Imaging, Diffusion Model, Inverse Problem

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sEYnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors in this work propose Principled Posterior Matching (PPM) as a way to approximate posterior learning in ill-posed inverse problems. The method was validated in a controlled toy setting and across different imaging problems, achieving comparable performance on fidelity metrics while exhibiting greater variability compared to state-of-the-art posterior learning methods. The main proposal of this work is a theoretical framework that distills pre-trained priors into a model that produces plausible alternative reconstructions with a single neural evaluation in its amortized form, using measurements instead of paired image datasets for reconstruction training, without the need for retraining on novel instances.

Critical strengths
  1. The proposed amortized formulation provides a model that is reusable and does not require multiple neural evaluations at inference time, in contrast to DPS and similar methods. Thus, repeated sampling becomes less expensive, which matters when the usage of learning the posterior is estimating parameters of the underlying distribution, i.e., variability, means, etc.

  2. The method doesn't need paired samples. PPM is able to train its reconstruction model on the available measurements given that the forward model is known, which is a fairly common setting in physics-informed scenarios, and does not require clean samples with matched observations. Thus, given a pre-trained prior, the method is able to use it to train an accurate posterior sampler.

  3. Connected to the previous point, the method uses the generated solution candidates directly and refines them during training, avoiding the likelihood approximation that DPS uses.

Critical weaknesses
  1. While the method successfully showcases that it increases the posterior variability, this is different than showing calibrated predictions. The reported metrics (PSNR, SSIM, etc.) show fidelity to a ground truth, while the proposed variability metric shows how diverse the reconstructions are. Nevertheless, this is not equivalent to the claim that the method offers calibrated uncertainty.

  2. The toy experiments are underused. While the plots show an obvious improvement in the outlined cases, I believe a proper metric should be introduced to evaluate the mismatch between the learned solution and the true posterior, for which an analytical solution is available.

  3. For the image experiments, they need a distribution-level evaluation as well, using a proper scoring rule. For instance, the energy score will help better separate the methods and clarify the benefits of this approach.

  4. Even though priors were trained specifically for each problem, I believe prior mismatch should be tested. This experiment would help clarify how sensitive the reconstruction quality and uncertainty estimates are to the distilled prior. It also increases the applicability of the method, as it could be that in some applications a closely matched prior is unavailable.

Questions for authors
  1. Could the authors obtain the analytical posteriors for the toy problems and measure energy scores for PPM and the baselines? Also include a plot of the exact posterior as a reference.
  2. Could the authors provide the energy score for the proposed method and the baselines in the image experiments using the same evaluation protocol as in the paper?
  3. Could you provide an experiment under mismatched priors? For instance, reconstructing Animal Faces-HQ (AFHQ) while training the prior on CelebA, and compare it against a domain-matched prior under the same circumstances.
  4. Could the authors provide an ablation on the soft data consistency term? For example, replacing the soft loss term with a data-consistency layer like the one proposed here: https://arxiv.org/html/1703.00555 for the undersampled MRI single-coil problem. The amortized reconstruction model should be trained only on the undersampled measurements, use the ground-truth images only for evaluation, and demonstrate the practical benefit of reconstruction training without paired clean targets. For simplicity, you can assume white Gaussian noise in the real and imaginary channels.
  5. Could the authors report the data consistency error using a normalized metric, for instance ∣∣A(x^)−y∣∣/∣∣y∣∣||A(\hat{x}) - y||/||y|| where x^\hat{x} is the reconstruction?

I would consider raising my score if these concerns are addressed.

6 comments

We thank the reviewer for their thoughtful feedback. We are pleased that the reviewer recognizes the efficiency of our reusable, single-pass amortized model, reconstruction training without paired clean targets given a pretrained prior, and direct evaluation of the measurement likelihood on candidate reconstructions. Below, we address the concerns raised.


W1. Sample diversity and uncertainty calibration

We agree that greater sample diversity does not establish calibrated uncertainty. PSNR and SSIM assess reconstruction fidelity, while our diversity metric and standard-deviation maps characterize sample variation. These quantities alone do not show that the learned distribution assigns the exact correct probabilities.

Our experiments support increased reconstruction diversity with competitive fidelity. We have clarified this scope in Sec. 4.2 and revised claims of accurate or reliable uncertainty quantification accordingly.

W2 & Q1. Quantitative evaluation on analytic toy posteriors

Thank you for suggesting a quantitative comparison against the exact posteriors. Both toy problems admit closed-form solutions: the prior is a Gaussian mixture, and the observation model is linear with Gaussian noise. Conditioning each component on the observed measurement and updating its weight by the corresponding marginal likelihood therefore gives the exact posterior mixture.

For Grid, the posterior retains all 16 components. The two dominant components are centered at approximately (-0.6892,-0.6892) and (0.6892,0.6892), each with posterior weight 0.49880. For Ring, the posterior retains all eight components, with weights proportional to exp⁡(−25sin⁡2θj)\exp(-25\sin^2\theta_j), where θj=jπ/4\theta_j=j\pi/4. The two dominant components are centered at (-2,0) and (2,0), each with weight 0.499996.

The table reports energy scores [1] (lower is better). The reference column is computed from samples drawn directly from the analytical posterior.

Toy problemExact posterior samples (reference)PPMRED-DiffPixel-RLSD
Grid0.932310.953051.305462.23420
Ring0.904011.006191.399911.52462

PPM achieves the lowest energy score among the three approximate methods in both problems. Its score is also closest to the sampling reference in both cases. These results complement the visual comparison by evaluating the predicted distributions rather than mode coverage alone.

W3 & Q2. Distribution-level evaluation on image tasks

We agree that sample diversity alone does not establish posterior accuracy or calibrated uncertainty. Following your recommendation, we evaluated CFID [2] and the energy score [1] for PPM-VI and the baselines on the same 128 FFHQ test observations, with 32 samples per observation.

Each entry below reports CFID64 / energy score, with lower values indicating better performance. CFID64 uses a fixed 64-dimensional PCA projection of the conditioning features, fitted on training data and shared across methods. Energy scores are normalized by the square root of the image dimensionality.

MethodSR ×8Motion deblurringBox inpainting
PPM-VI112.9722 / 0.04418990.1099 / 0.02949754.0576 / 0.037347
RED-Diff115.0422 / 0.045491104.2180 / 0.03369255.1678 / 0.046542
DPS102.7897 / 0.04953589.4741 / 0.03975154.0827 / 0.039355

PPM-VI achieves the lowest energy score on all three tasks. The CFID64 results are more mixed: DPS scores lower on ×8 super-resolution and motion deblurring, while PPM-VI and DPS obtain nearly identical values on box inpainting (54.06 and 54.08, respectively).

In the reconstruction results reported in Table 2, DPS also produces diverse samples but its PSNR and SSIM are lower than those of PPM-VI on all three tasks. Overall, these results suggest that PPM-VI strikes a favorable balance between sample diversity and reconstruction fidelity across the three tasks.

W4 & Q3. Sensitivity to prior mismatch

We tested prior mismatch by using an ImageNet-trained 256×256256\times256 diffusion prior to reconstruct FFHQ 256×256256\times256 images. The table reports results with the ImageNet prior on the same three restoration tasks. Each entry reports energy score / normalized measurement residual (lower is better for both).

Method×8 super-resolutionMotion deblurringBox inpainting
DPS0.046522 / 0.1220780.054279 / 0.1199930.044718 / 0.107772
RED-Diff0.048437 / 0.0324150.036700 / 0.1130460.046429 / 0.084262
PPM-VI0.040516 / 0.0580700.029387 / 0.1097960.044625 / 0.027623

The effect of prior mismatch is task-dependent. PPM-VI still has the lowest energy score among the three methods on all tasks and the lowest measurement residual on deblurring and inpainting, while RED-Diff has the lowest residual on super-resolution.

Q4. Soft data consistency and an MRI ablation

Thank you for suggesting this ablation.

  • Role of a DC layer. For single-coil Cartesian MRI, the suggested layer [3] directly corrects the acquired k-space coefficients. This explicit correction should improve measurement consistency. Given a pretrained prior, it can be incorporated into PPM's reconstruction network while training only on undersampled measurements, with fully sampled ground truth reserved for evaluation.

  • Treatment of measurement noise. Under white Gaussian noise in the real and imaginary channels, our noise-weighted quadratic data term is the negative log-likelihood up to a constant. The suggested layer supports finite-weight correction; hard replacement is its noiseless limit. Hard replacement would copy measurement noise and eliminate sample variation at acquired coefficients. Finite-strength correction should therefore be more appropriate for noisy measurements. A DC layer alone is not generally equivalent to retaining the likelihood term, and its effect on reconstruction quality and posterior accuracy requires empirical evaluation. This ablation remains incomplete due to limited compute alongside the other additional experiments; we will add results to the revision if available.

Q5. Normalized data-consistency error

We report the normalized data consistency error ∣∣A(x^)−y∣∣∣∣y∣∣\frac{||A(\hat x)-y||}{||y||}, averaged over the same N=128 FFHQ test observations and K=8 samples per observation

MethodSR ×8Motion deblurringBox inpainting
RED-Diff0.0679130.1001450.085103
DPS0.1147720.1177390.126857
PPM-VI0.0368430.1132910.025371

PPM-VI achieves a lower normalized data consistency error than DPS on all three tasks. On ×8 super-resolution and box inpainting, it also reduces the error relative to RED-Diff by approximately 46% and 70%, respectively, with the largest gain on box inpainting. On motion deblurring, RED-Diff achieves the lowest error (0.1001), followed by PPM-VI (0.1133) and DPS (0.1177).


Reference

[1] Tilmann Gneiting and Adrian E. Raftery. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association, 2007.

[2] Michael Soloveitchik, et al. Conditional Frechet Inception Distance. arXiv:2103.11521, 2021.

[3] Jo Schlemper, et al. A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction. arXiv:1703.00555, 2017.

I appreciate the authors response. Most of my concerns have been solved. I will wait for the data consistency experiments to be updated as well as the rest of the experiments in the manuscript to update my score.

Reviewer_6wbdOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce PPM, which provides a variational inference framework for posterior sampling using diffusion model priors. This avoids the approximations needed for guidance-based methods as well as the point-estimate/repulsion approximations of variational methods. They propose two different approaches: particle-based inference for a single measurement, as well as a amortized learned network for many measurements. To do so, they train a second diffusion model qq that represents the noised scores of the variational distribution and uses the pathwise KL to evaluate the prior KL term in the variational objective. Experiments on a wide range of scientific inverse problems demonstrate that PPM maintains reconstruction quality but increases sample diversity compared to baselines.

Critical strengths

Diffusion models are strong priors for inverse problems but their reliance on bias-inducing approximations during guidance or MCMC can make them difficult to trust for scientific inverse problems, particularly for metrics like uncertainty. The paper proposes that variational inference, which has traditionally caused mode-seeking or approximate repulsion behaviors, can be modified to avoid this bias, which is an interesting perspective.

Critical weaknesses

For the particle-based sampler, the noisy score of qtq_t is analytic, as it is just a Gaussian KDE of a few delta points, it is unclear why it should be learned which introduces both complexity and possible score errors. I doubt it would exhibit the generalization properties of diffusion models with such few particles as well.

The parameterization of the amortized network has a strange design choice, where the posterior of measurement yy is sampled using g(y+n),n∼N(0,hI)g(y + n), n \sim N(0, hI). This implicitly assumes that nearby measurements have similar posteriors, which may not be the case in highly multimodal or ill-conditioned inverse problems.

Missing baseline: how does it compare to "Score-Based Diffusion Models as Principled Priors for Inverse Imaging", which also presents a way to learn variational posteriors from diffusion models in an unbiased way? While that paper uses the less expressive normalizing flow and requires computing log probabilities, they still use the same variational inference approach.

Questions for authors

Why is learning this score of the particle based case necessary here?

For the particle-based case, other particle based samplers (i.e. SVGD) suffer from curse of dimensionality. At low diffusion noise tt, the distribution qtq_t is a set of disjoint Gaussians due to finite samples, yet the KL path weighting is strong in this regime. Does this mismatch cause issues, on analytically known posteriors?

5 comments

We thank the reviewer for their thoughtful feedback. We are pleased that the reviewer recognizes the value of revisiting variational inference for diffusion-based inverse problems, particularly its potential to improve posterior estimation for scientific imaging. Below, we address the concerns raised.


W1 & Q1. Why learn the variational score?

The variational score is analytic for a fixed Gaussian particle mixture. Here, we use a learned score for two reasons:

  • Pretrained prior with few particles. We initialize the auxiliary network from the pretrained prior, inheriting image features and denoising structure learned from a much larger training set. These features are reused when fitting noisy reconstructions across noise levels, providing an inductive bias that can guide score estimation in regions sparsely covered by the current particles. ProlificDreamer [1] discusses a similar benefit even with one particle and reports improvements over SDS [2] (Sec. 3.3).

  • Distribution-level training for amortized inference. A stochastic reconstruction network induces an implicit distribution whose score is generally unavailable in closed form. The auxiliary network estimates this score from generated samples, allowing the Fisher update to account for the evolving reconstruction distribution during posterior matching. DMD [3] similarly learns the generator's score to enable distribution-matching updates (Sec. 3.2).

The possible benefit with few particles comes from the pretrained parameterization and shared score learning, not from having fewer particles itself. We have clarified this distinction and the analytic mixture-score alternative in Sec. 3.2.

W2. Why perturb the measurement input?

  • Introducing a variational distribution. For a fixed measurement yy, independent perturbations in x=gϕ(y+η)x=g_\phi(y+\eta), η∼N(0,hI)\eta\sim\mathcal{N}(0,hI), produce samples from qϕ(x∣y)q_\phi(x\mid y). Without this randomness, the network returns a deterministic reconstruction. The likelihood remains evaluated against the original yy, so the samples are optimized for the same observation.

  • Measurement coupling in the evaluated settings. Nearby conditions are coupled, so this parameterization is most appropriate when posterior variation is moderate at the chosen perturbation scale. Our restoration results support its practical use in the tested settings. DAVI [4] follows the same stochastic-input principle and also reports improved LPIPS and FID with input perturbation (Table 5).

W3. Comparison with flow-based variational inference

We thank the reviewer for highlighting this closely related work. We agree that Feng et al. [5] already establish a principled variational approach using diffusion priors. Their implementation optimizes a normalizing-flow posterior using prior log-probabilities and gradients computed through a probability-flow ODE. PPM contributes an alternative treatment of the prior KL through KL dissipation and score projection, supporting implicit posterior parameterizations without explicit density evaluation. We have clarified this relationship in Sec. 2.2 and qualified our unbiased-gradient claim by its exact-score assumptions in Appendix A.

The original paper demonstrates image reconstruction at 32×32 resolution, without reporting a 256×256 FFHQ benchmark. Its supplementary material also identifies log-probability computation as the main optimization bottleneck, reporting 30–200 seconds per optimization step on v4-8 TPUs, with convergence generally requiring 5,000–10,000 steps. Once fitted, however, the flow supports fast sampling (Supplementary Sec. A.2 of [5]).

To investigate its applicability to our setting, we implemented a 256×256 convolutional RealNVP adaptation with the same FFHQ diffusion prior used in our comparisons. We evaluated five observations per task and eight reconstructions per observation. Under a budget of 1,024 optimization steps, optimization consumed approximately 15–22 GPU-hours per observation across our A40/A800 runs. Reconstruction quality remained unsatisfactory: mean PSNR was 13.95, 11.03, and 19.96 dB for ×8 super-resolution, motion deblurring, and box inpainting, respectively, with substantial noise or incomplete filling of the masked region. These results document the computational and optimization difficulties encountered in our adaptation; they do not establish the fully converged performance of the original method.

Q2. Finite particles at low diffusion noise

We agree that finite-particle resolution matters at low diffusion noise. For Gaussian perturbation, the particle distribution is

qϕ,t(x∣y)=1K∑k=1KN(x;αtμk,σt2I).q_{\phi,t}(x\mid y)=\frac{1}{K}\sum_{k=1}^{K}\mathcal{N}(x;\alpha_t\mu_k,\sigma_t^2 I).

At low noise, component overlap can become negligible, making the variational score predominantly local.

Our formulation makes the integration interval [tmin⁡,T][t_{\min},T] explicit. Larger diffusion times increase component overlap, while the lower cutoff and time weighting determine how strongly small-noise behavior enters optimization. Theorem 1 concerns the gradient of the specified positive-time objective: it does not guarantee that a finite particle set accurately represents the full posterior. The toy visualization in Fig. 1 illustrates mode coverage in the displayed examples, but does not establish robustness to small tmin⁡t_{\min} or insufficient particles.

To assess this empirically, we evaluated PPM on the Grid and Ring problems, whose exact posteriors are available analytically from their Gaussian-mixture priors and linear Gaussian observation models. PPM achieves energy scores (lower is better) of 0.95305 on Grid and 1.00619 on Ring, compared with 0.93231 and 0.90401, respectively, for samples drawn directly from the exact posteriors. PPM also obtains lower energy scores than RED-Diff and Pixel-RLSD on both problems. These results provide quantitative support for the finite-particle approximation in the evaluated settings. They do not, by themselves, establish robustness to smaller noise cutoffs, fewer particles, or higher dimensions.


Reference

[1] Zhengyi Wang, et al. ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation. NeurIPS, 2023.

[2] Ben Poole, et al. DreamFusion: Text-to-3D using 2D Diffusion. ICLR, 2023.

[3] Tianwei Yin, et al. One-step Diffusion with Distribution Matching Distillation. CVPR, 2024.

[4] Sojin Lee, et al. Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problems. ECCV, 2024.

[5] Berthy T. Feng, et al. Score-Based Diffusion Models as Principled Priors for Inverse Imaging. ICCV, 2023.

Reviewer_6wbdOpenReview ↗

Response to Authors

I thank the authors for their thorough response.

While the previous work is related, I am wondering does the learned variational score in the particle case actually demonstrate any improvement over the analytic score for posterior inference, and how crucial is the engineering i.e. precise early stopping of training to prevent memorization? Also, to clarify W2, I meant why not parameterize as g(y,n),n∼N(0,I)g(y, n), n \sim N(0,I) to decouple posterior uncertainty with measurement noise. I'm happy to raise my score if these questions are answered.

Reviewer_7jPPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses variational inference for inverse problems with a diffusion prior where the obstacle is the prior KL term, whose gradient needs the score of the variational distribution. The authors rewrite that KL via the KL-dissipation identity, as an integrated Fisher divergence between the diffused variational and prior distributions, and use the score-projection identity to obtain a pathwise gradient that only requires an auxillary score network trained on samples from the variational distribution. The authors show their method with experiments on FFHQ/ImageNet restoration, black-hole imaging, and microscopy. Their samples show more diversity than RED-Diff and DAVI at comparable or better PSNR.

Critical strengths

S1) Tying the Fisher objective to the exact prior KL with the terminal term stated rather than dropped is cleaner than the integral KL used by DAVI.

S2) One objective serves both per-observation particles and an amortized network, with no likelihood score approximation.

S3) The paper overall is well-written and easy to follow.

Critical weaknesses

W1) The "unbiased/exact" framing contradicts the paper's own caveats. The gradient is unbiased only for exact scores. These claims should be relaxed.

W2) Posterior quality is asserted, but it is not measured well. The diversity score is not calibrated. I suggest the authors look at CFID [1], which aims to directly posterior approximation accuracy.

W3) Reporting is fairly thin. The test set is “32 randomly selected images” in one sentence and “1,000 observations” in the next; ImageNet and Gaussian-denoising numbers are promised but absent from tables; no error bars. Particle PPM trains an auxiliary score network per observation and its cost relative to RED-Diff or DPS is not reported. Baselines run at “original defaults” on tasks (×8 SR, 61×61 motion blur) those defaults were not tuned for.

W4) Some numbers need explanation. PPM-VI box inpainting at 28.73 dB / 0.97 SSIM for a 128×128 center mask is 4 dB above every baseline and above PPM’s own amortized mode (25.26 / 0.88), while on the other two tasks the two modes are within 1 dB and RED-Diff beats PPM-VI on motion-deblurring PSNR.

Questions for authors

Q1) What is the measurement noise for each task, and what are λ_data, λ_aux, ρ(t), t_min, h, K and the iteration counts?

Q2) What is the per-observation compute (iterations, NFEs, wall-clock) of particle PPM versus RED-Diff and DPS, and the training cost of amortized PPM versus DAVI?

Q3) Why is PPM-VI 4 dB above PPM-AI and all baselines on box inpainting only, and how were the particles initialized there?

Q4) Can you compare the sample sets to a reference posterior with a distributional metric, and report calibration or coverage of the std maps?

Q5) Were the Table 2 numbers computed on 32 or 1,000 images, and where are the ImageNet and Gaussian-denoising tables?

4 comments

We thank the reviewer for their thoughtful feedback. We are pleased that the reviewer recognizes the clarity of our presentation, the explicit connection between our Fisher objective and the prior KL, and the unified treatment of particle-based and amortized inference. Below, we address the concerns raised.


W1. Scope of the unbiased-gradient claim

We agree that scores learned by diffusion models are approximations to the exact scores. Theorem 1 gives an unbiased Monte Carlo estimator of the gradient only under the exact-score assumption. We have clarified this point in Appendix A and relaxed the corresponding claims in the manuscript.

W2 & Q4. Posterior quality and calibration

We agree that sample diversity alone does not establish posterior accuracy or calibrated uncertainty. Following your recommendation, we evaluated CFID [1] and the energy score [2] for PPM-VI and the baselines on the same 128 FFHQ test observations, with 32 samples per observation.

Each entry below reports CFID64 / energy score, with lower values indicating better performance. CFID64 uses a fixed 64-dimensional PCA projection of the conditioning features, fitted on training data and shared across methods. Energy scores are normalized by the square root of the image dimensionality.

MethodSR ×8Motion deblurringBox inpainting
PPM-VI112.9722 / 0.04418990.1099 / 0.02949754.0576 / 0.037347
RED-Diff115.0422 / 0.045491104.2180 / 0.03369255.1678 / 0.046542
DPS102.7897 / 0.04953589.4741 / 0.03975154.0827 / 0.039355

PPM-VI achieves the lowest energy score on all three tasks. The CFID64 results are more mixed: DPS scores lower on ×8 super-resolution and motion deblurring, while PPM-VI and DPS obtain nearly identical values on box inpainting (54.06 and 54.08, respectively).

In the reconstruction results reported in Table 2, DPS also produces diverse samples but its PSNR and SSIM are lower than those of PPM-VI on all three tasks. Overall, these results suggest that PPM-VI strikes a favorable balance between sample diversity and reconstruction fidelity across the three tasks.

W3 & Q1 & Q5. Experimental reporting, and baseline settings

We jointly optimize K=8 particles in PPM-VI. In the new FFHQ evaluation, ×8 super-resolution, motion deblurring, and box inpainting all use Gaussian measurement noise with σy=0.05\sigma_y=0.05 on the [-1,1] image scale; for inpainting, noise is added only to observed pixels.

For the new PPM-VI runs, λdata=1/(2σy2)=200\lambda_{\rm data}=1/(2\sigma_y^2)=200 for the sum of squared residuals; For the learned auxiliary score, we use the standard unweighted DDPM noise-prediction loss, which corresponds to λaux(t)=σt2=1−αˉt\lambda_{\rm aux}(t)=\sigma_t^2=1-\bar\alpha_t in the score-space objective of Eq. (10), where σt\sigma_t is the forward diffusion noise standard deviation. We sample DDPM indices i=0,…,999i=0,\ldots,999 with ρi=0.9wi/∑jwj+0.1/1000\rho_i=0.9w_i/\sum_j w_j+0.1/1000, where wi=biαˉi/(1−αˉi)w_i=b_i\bar\alpha_i/(1-\bar\alpha_i) and bib_i are trapezoidal weights in −log⁡αˉi-\log\bar\alpha_i. The tmint_{min} is 0, corresponding to diffusion standard deviation ≈0.01\approx 0.01. The separately evaluated PPM-AI checkpoints used h=0.05h=0.05 at training and inference time.

The metrics in Table 2 are averaged over 1,000 FFHQ test images. The CFID and energy-score results reported above are from an additional evaluation on 128 FFHQ test observations, with 32 samples per observation.


Reference

[1] Michael Soloveitchik, et al. Conditional Frechet Inception Distance. arXiv:2103.11521, 2021.

[2] Tilmann Gneiting and Adrian E. Raftery. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association, 2007.

Q2. Computational cost

We report costs for the additional evaluation on 128 FFHQ observations at 256×256 resolution, with eight reconstructions per observation. The following timings are means ± standard deviations on one NVIDIA A800 GPU, using microbatches of four

MethodIterationsPrior NFEs per observation, K=8Time (s)
PPM-VI1,0008,000109.7 ± 2.4
RED-Diff1,0008,00090.9 ± 1.2
DPS1,0008,000167.2 ± 1.7

For amortized inference, training is performed once per task. Separate, fully logged retraining runs used 30,000 main training iterations and batch size eight:

TaskPPM-AI recorded GPU-hoursDAVI recorded GPU-hours
SR ×85.745.83
Motion deblurring5.625.82
Box inpainting5.685.75

W4 & Q3. Performance gap on inpainting

The gap on inpainting can be understood from three aspects:

  • Why inpainting? Inpainting directly observes the pixels outside the mask, thus only the missing region needs to be inferred. We initialize PPM-VI and RED-Diff particles from the masked image, so the observed region starts at its measured values. This provides a favorable starting point compared with sampling procedures that reconstruct the full image from noise. Super-resolution and deblurring do not offer the same directly observed image region.

  • PPM-VI vs. other particle-based baselines. RED-Diff uses a single-particle, Dirac variational parameterization per run. PPM-VI jointly optimizes multiple particles, allowing them to represent different modes of the completion distribution. This is useful in inpainting, where a fixed visible context can admit multiple plausible solutions. Furthermore, we use empirical minimum Bayes risk selection to choose the particle closest to the ensemble mean, yielding a representative reconstruction.

  • PPM-VI vs. PPM-AI. PPM-VI directly optimizes the initialized particles for each observation. PPM-AI maps the measurement to a full image through an amortized network and does not inherently preserve pixels outside the mask. It may therefore introduce errors even in the observed region, in addition to errors in the completion. This difference can contribute to the larger VI–AI gap on inpainting.

My thanks to the authors. I do feel that my concerns have been addressed. Therefore, I raise my overall score to 3.

Thank you for your kind response!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.