- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors in this work propose Principled Posterior Matching (PPM) as a way to approximate posterior learning in ill-posed inverse problems. The method was validated in a controlled toy setting and across different imaging problems, achieving comparable performance on fidelity metrics while exhibiting greater variability compared to state-of-the-art posterior learning methods. The main proposal of this work is a theoretical framework that distills pre-trained priors into a model that produces plausible alternative reconstructions with a single neural evaluation in its amortized form, using measurements instead of paired image datasets for reconstruction training, without the need for retraining on novel instances.
Critical strengths
-
The proposed amortized formulation provides a model that is reusable and does not require multiple neural evaluations at inference time, in contrast to DPS and similar methods. Thus, repeated sampling becomes less expensive, which matters when the usage of learning the posterior is estimating parameters of the underlying distribution, i.e., variability, means, etc.
-
The method doesn't need paired samples. PPM is able to train its reconstruction model on the available measurements given that the forward model is known, which is a fairly common setting in physics-informed scenarios, and does not require clean samples with matched observations. Thus, given a pre-trained prior, the method is able to use it to train an accurate posterior sampler.
-
Connected to the previous point, the method uses the generated solution candidates directly and refines them during training, avoiding the likelihood approximation that DPS uses.
Critical weaknesses
-
While the method successfully showcases that it increases the posterior variability, this is different than showing calibrated predictions. The reported metrics (PSNR, SSIM, etc.) show fidelity to a ground truth, while the proposed variability metric shows how diverse the reconstructions are. Nevertheless, this is not equivalent to the claim that the method offers calibrated uncertainty.
-
The toy experiments are underused. While the plots show an obvious improvement in the outlined cases, I believe a proper metric should be introduced to evaluate the mismatch between the learned solution and the true posterior, for which an analytical solution is available.
-
For the image experiments, they need a distribution-level evaluation as well, using a proper scoring rule. For instance, the energy score will help better separate the methods and clarify the benefits of this approach.
-
Even though priors were trained specifically for each problem, I believe prior mismatch should be tested. This experiment would help clarify how sensitive the reconstruction quality and uncertainty estimates are to the distilled prior. It also increases the applicability of the method, as it could be that in some applications a closely matched prior is unavailable.
Questions for authors
- Could the authors obtain the analytical posteriors for the toy problems and measure energy scores for PPM and the baselines? Also include a plot of the exact posterior as a reference.
- Could the authors provide the energy score for the proposed method and the baselines in the image experiments using the same evaluation protocol as in the paper?
- Could you provide an experiment under mismatched priors? For instance, reconstructing Animal Faces-HQ (AFHQ) while training the prior on CelebA, and compare it against a domain-matched prior under the same circumstances.
- Could the authors provide an ablation on the soft data consistency term? For example, replacing the soft loss term with a data-consistency layer like the one proposed here: https://arxiv.org/html/1703.00555 for the undersampled MRI single-coil problem. The amortized reconstruction model should be trained only on the undersampled measurements, use the ground-truth images only for evaluation, and demonstrate the practical benefit of reconstruction training without paired clean targets. For simplicity, you can assume white Gaussian noise in the real and imaginary channels.
- Could the authors report the data consistency error using a normalized metric, for instance where is the reconstruction?
I would consider raising my score if these concerns are addressed.
6 comments
W4 & Q3. Sensitivity to prior mismatch
We tested prior mismatch by using an ImageNet-trained diffusion prior to reconstruct FFHQ images. The table reports results with the ImageNet prior on the same three restoration tasks. Each entry reports energy score / normalized measurement residual (lower is better for both).
| Method | ×8 super-resolution | Motion deblurring | Box inpainting |
|---|---|---|---|
| DPS | 0.046522 / 0.122078 | 0.054279 / 0.119993 | 0.044718 / 0.107772 |
| RED-Diff | 0.048437 / 0.032415 | 0.036700 / 0.113046 | 0.046429 / 0.084262 |
| PPM-VI | 0.040516 / 0.058070 | 0.029387 / 0.109796 | 0.044625 / 0.027623 |
The effect of prior mismatch is task-dependent. PPM-VI still has the lowest energy score among the three methods on all tasks and the lowest measurement residual on deblurring and inpainting, while RED-Diff has the lowest residual on super-resolution.
Q4. Soft data consistency and an MRI ablation
Thank you for suggesting this ablation.
-
Role of a DC layer. For single-coil Cartesian MRI, the suggested layer [3] directly corrects the acquired k-space coefficients. This explicit correction should improve measurement consistency. Given a pretrained prior, it can be incorporated into PPM's reconstruction network while training only on undersampled measurements, with fully sampled ground truth reserved for evaluation.
-
Treatment of measurement noise. Under white Gaussian noise in the real and imaginary channels, our noise-weighted quadratic data term is the negative log-likelihood up to a constant. The suggested layer supports finite-weight correction; hard replacement is its noiseless limit. Hard replacement would copy measurement noise and eliminate sample variation at acquired coefficients. Finite-strength correction should therefore be more appropriate for noisy measurements. A DC layer alone is not generally equivalent to retaining the likelihood term, and its effect on reconstruction quality and posterior accuracy requires empirical evaluation. This ablation remains incomplete due to limited compute alongside the other additional experiments; we will add results to the revision if available.
Q5. Normalized data-consistency error
We report the normalized data consistency error , averaged over the same N=128 FFHQ test observations and K=8 samples per observation
| Method | SR ×8 | Motion deblurring | Box inpainting |
|---|---|---|---|
| RED-Diff | 0.067913 | 0.100145 | 0.085103 |
| DPS | 0.114772 | 0.117739 | 0.126857 |
| PPM-VI | 0.036843 | 0.113291 | 0.025371 |
PPM-VI achieves a lower normalized data consistency error than DPS on all three tasks. On ×8 super-resolution and box inpainting, it also reduces the error relative to RED-Diff by approximately 46% and 70%, respectively, with the largest gain on box inpainting. On motion deblurring, RED-Diff achieves the lowest error (0.1001), followed by PPM-VI (0.1133) and DPS (0.1177).
Reference
[1] Tilmann Gneiting and Adrian E. Raftery. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association, 2007.
[2] Michael Soloveitchik, et al. Conditional Frechet Inception Distance. arXiv:2103.11521, 2021.
[3] Jo Schlemper, et al. A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction. arXiv:1703.00555, 2017.
I appreciate the authors response. Most of my concerns have been solved. I will wait for the data consistency experiments to be updated as well as the rest of the experiments in the manuscript to update my score.
We thank the reviewer for their thoughtful feedback. We are pleased that the reviewer recognizes the efficiency of our reusable, single-pass amortized model, reconstruction training without paired clean targets given a pretrained prior, and direct evaluation of the measurement likelihood on candidate reconstructions. Below, we address the concerns raised.
W1. Sample diversity and uncertainty calibration
We agree that greater sample diversity does not establish calibrated uncertainty. PSNR and SSIM assess reconstruction fidelity, while our diversity metric and standard-deviation maps characterize sample variation. These quantities alone do not show that the learned distribution assigns the exact correct probabilities.
Our experiments support increased reconstruction diversity with competitive fidelity. We have clarified this scope in Sec. 4.2 and revised claims of accurate or reliable uncertainty quantification accordingly.
W2 & Q1. Quantitative evaluation on analytic toy posteriors
Thank you for suggesting a quantitative comparison against the exact posteriors. Both toy problems admit closed-form solutions: the prior is a Gaussian mixture, and the observation model is linear with Gaussian noise. Conditioning each component on the observed measurement and updating its weight by the corresponding marginal likelihood therefore gives the exact posterior mixture.
For Grid, the posterior retains all 16 components. The two dominant components are centered at approximately (-0.6892,-0.6892) and (0.6892,0.6892), each with posterior weight 0.49880. For Ring, the posterior retains all eight components, with weights proportional to exp(−25sin2θj), where θj=jπ/4. The two dominant components are centered at (-2,0) and (2,0), each with weight 0.499996.
The table reports energy scores [1] (lower is better). The reference column is computed from samples drawn directly from the analytical posterior.
PPM achieves the lowest energy score among the three approximate methods in both problems. Its score is also closest to the sampling reference in both cases. These results complement the visual comparison by evaluating the predicted distributions rather than mode coverage alone.
W3 & Q2. Distribution-level evaluation on image tasks
We agree that sample diversity alone does not establish posterior accuracy or calibrated uncertainty. Following your recommendation, we evaluated CFID [2] and the energy score [1] for PPM-VI and the baselines on the same 128 FFHQ test observations, with 32 samples per observation.
Each entry below reports CFID64 / energy score, with lower values indicating better performance. CFID64 uses a fixed 64-dimensional PCA projection of the conditioning features, fitted on training data and shared across methods. Energy scores are normalized by the square root of the image dimensionality.
PPM-VI achieves the lowest energy score on all three tasks. The CFID64 results are more mixed: DPS scores lower on ×8 super-resolution and motion deblurring, while PPM-VI and DPS obtain nearly identical values on box inpainting (54.06 and 54.08, respectively).
In the reconstruction results reported in Table 2, DPS also produces diverse samples but its PSNR and SSIM are lower than those of PPM-VI on all three tasks. Overall, these results suggest that PPM-VI strikes a favorable balance between sample diversity and reconstruction fidelity across the three tasks.