- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PIVOT, a framework for diagnosing misspecification of diffusion priors by introducing an interpretable discrepancy variable that indexes a family of perturbed priors . A conditional diffusion model with label dropout is used so that posterior inference over the discrepancy can be approximately amortized from a single posterior-sampling run. Experiments on a few cases like Gaussian toy problem, travel-time tomography, and M87* black-hole imaging are provided to justify the effectiveness of proposed methodology.
Critical strengths
The formulation is intuitive and useful. Instead of only detecting that a prior is mismatched, the method asks which scientifically meaningful direction of misspecification is supported by the data. The amortized inference strategy is also computationally appealing.
Critical weaknesses
The reviewer's main concern is that the inferred discrepancy posterior relies on several approximations, including approximate posterior sampling and ELBO-based density ratios. The Gaussian example validates these approximations in a controlled setting, but there is no direct calibration against a non-amortized evidence computation on the realistic experiments.
The M87* conclusion also appears somewhat stronger than what the experiment alone establishes. A high discrepancy relative to simulated GRMHD observations indicates mismatch along the proposed compact-emission direction, but other forms of prior misspecification could potentially be absorbed by the same discrepancy family.
The related-work discussion could also be potentially enriched by connecting PIVOT to recent work [1-7] on inference/test-time adaptation and scaling with diffusion priors, which can all be thought of as ensemble-based posterior inference methods. Therefore, the authors are strongly encouraged to expand the discussion of related work on inference/test-time methods with diffusion-based priors, which helps enhance quality of the manuscript.
Questions for authors
-
Would it be possible for the authors to compare the amortized against a non-amortized reference obtained by solving the inverse problem separately for several fixed values of ?
-
How robust is the M87* compact-emission conclusion to alternative discrepancy families? Could another generic prior mismatch also lead to a large inferred emission discrepancy?
References:
[1] Singhal, R., Horvitz, Z., Teehan, R., Ren, M., Yu, Z., McKeown, K., & Ranganath, R. (2025). A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.06848.
[2] Skreta, M., Akhound-Sadegh, T., Ohanesian, V., Bondesan, R., Aspuru-Guzik, A., Doucet, A., ... & Neklyudov, K. (2025). Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503.02819.
[3] Uehara, M., Zhao, Y., Wang, C., Li, X., Regev, A., Levine, S., & Biancalani, T. (2025). Inference-time alignment in diffusion models with reward-guided generation: Tutorial and review. arXiv preprint arXiv:2501.09685.
[4] He, J., Hernández-Lobato, J. M., Du, Y., & Vargas, F. (2025). Rne: a plug-and-play framework for diffusion density estimation and inference-time control. arXiv e-prints, arXiv-2506.
[5] Wang, C., Wang, W., Ren, Y., Blanchet, J., & Lu, Y. (2026, May). Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures. In Forty-third International Conference on Machine Learning.
[6] Ren, Y., Gao, W., Ying, L., Rotskoff, G., & Han, J. (2026, April). Driftlite: Lightweight drift control for inference-time scaling of diffusion models. In International Conference on Learning Representations (Vol. 2026, pp. 76664-76702).
[7] Chen, H., Ren, Y., Min, M. R., Ying, L., & Izzo, Z. (2025). Solving inverse problems via diffusion-based priors: An approximation-free ensemble sampling approach. arXiv preprint arXiv:2506.03979.
3 comments
Response
The reviewer would like to thank the authors for providing a detailed response and uploading the revised manuscript, which has satisfactorily addressed most of the concerns raised in the original review. Consequently, the reviewer would be glad to raise the rating from 2 to 3.
Comment by Authors
Thank you for the thoughtful response and considering our revised manuscript!
Author Response to 1Wdz (1)
We thank the reviewer for recognizing our work as “intuitive and useful” and “computationally appealing”. We address their concerns below.
W1, Q1. For the ELBO concern, see W1 response for Reviewer Np1P where we extensively validate the ELBO approximation and find that it matches both the highly discretized, integrated PF-ODE density and recovers the generating ground truth condition.
Sampling p(x∣y,a) for a fixed a does not directly provide the evidence p(y∣a) which is required for constructing a non-amortized reference p(a∣y), and obtaining reliable evidence estimates for diffusion priors is difficult. Existing evidence estimators such as DiME [1] have not been extended to the posterior sampling method we use in our paper and also demonstrate that classical estimators lead to biased results on diffusion priors. Thus, the suggested experiment would require significant extension and validation of a new estimator. To address the concern, we run experiments testing the amortization assumption itself: whether the amortized head p(x∣∅) represents the uniform mixture ∫p(x∣a)p(a)da as suggested by reviewer 5LP2. If this holds, the amortized pamort(a∣y) is mathematically identical to the non-amortized pnonamort(a∣y); otherwise, it tells us how much amortization error can distort a∣y:
\propto p_{\mathrm{nonamort}}(a\mid y) \mathbb{E}_{x\sim p(x\mid y,a)} \left[ \frac{p(x\mid\varnothing)} {\int p(x\mid a')p(a')\\,da'} \right].$$ See $\textbf{W2}$ response for Np1P for results: $p(x \mid \varnothing)$ closely matches $\int p(x \mid a) p(a) da$ in both image-space FID up to finite-sample variability as well as condition-space $W_1$, providing empirical support that the amortization is valid. Finally, the synthetic Gaussian experiments include the effects of the approximate posterior sampling method, yet still closely match the analytic evidence. We will add these results to the final version of the paper. [1] Wang et al. Sample-efficient evidence estimation of score based priors for model selection, ICLR 2026. $\textbf{W2, Q2}$. We agree that the emission family is not the unique explanation; our method tests whether a specific physically motivated hypothesis is more strongly supported by the data via the Bayesian evidence, in which case the M87* observations result in greater support for compact emission discrepancies than in-distribution GRMHD. For alternate discrepancies, we already demonstrate that elliptically perturbed measurements lead to similarly low emission discrepancy. While this does not account for every possible discrepancy, it does confirm that a high emission discrepancy does not always occur from GRMHD prior mismatch. Many discrepancies, such as variations in ring diameter (35uas - 50uas), spin (-0.94, -0.5, 0, 0.5, 0.94), or inclination (0 to 180 deg) are already captured in the ordinary variability of our GRMHD training dataset. On the other hand, additional ellipticity and off-ring emission are two plausible, physically motivated, and actively studied [2-3] open questions that are not captured by our GRMHD simulations, which is why they were specifically chosen. Therefore, while another mismatch could lead to a large inferred emission discrepancy, these two discrepancies are particularly of interest for M87*. [2] Dahale et al. Origin of the ring ellipticity in the black hole images of M87∗, A&A 2025 [3] Muller et al. Probing jet base emission of M87* with the 2021 Event Horizon Telescope observations, A&A 2026 $\textbf{W3}$. We thank the reviewer for pointing out these references, and we will expand upon the discussion of diffusion posterior inference in the final version.