acceptodds
Under review as a conference paper at ICLR 2027

When the Prior Is Wrong: Probing Misspecified Diffusion Priors in Inverse Problems

Abstract

While diffusion priors generate high-quality posterior samples across many scientific inverse problems, they are often trained on purely simulated data, thus inheriting the errors and biases of these underlying sources. Many approaches to adapting diffusion priors from a single observation require sufficient conflict between the data and the prior to successfully adapt and do not allow specific misspecification hypotheses to be tested. On the other hand, existing out-of-distribution (OOD) detection methods determine whether the observation is compatible with a fixed prior, rather than specifying which assumptions to relax or by how much. To address these issues, we propose PIVOT: Prior Inference Via Observation-driven Transformations, which formulates diffusion prior misspecification as inference over an underlying latent variable representing a scientifically plausible discrepancy. By expanding the prior along these plausible directions of misspecification, PIVOT infers a posterior over the strength of each proposed discrepancy given the observation. This process can be efficiently amortized with just one posterior sampling pass using a conditional diffusion model. We also demonstrate how to determine if a proposed discrepancy is inconclusive because the measurement is too noisy or ill-posed. We validate PIVOT on seismic imaging and real-world black hole imaging problems, where we find that OOD observations generally lead to larger perturbation strengths when the proposed discrepancy family captures relevant components of the mismatch, as long as the discrepancy is visible to the measurement. PIVOT thereby provides an interpretable probe into how real-world priors could be misspecified.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
diffusion models, inverse problems, model selection, robust bayes, computational imaging

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_1WdzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PIVOT, a framework for diagnosing misspecification of diffusion priors by introducing an interpretable discrepancy variable aa that indexes a family of perturbed priors p(x∣a)p(x\mid a). A conditional diffusion model with label dropout is used so that posterior inference over the discrepancy can be approximately amortized from a single posterior-sampling run. Experiments on a few cases like Gaussian toy problem, travel-time tomography, and M87* black-hole imaging are provided to justify the effectiveness of proposed methodology.

Critical strengths

The formulation is intuitive and useful. Instead of only detecting that a prior is mismatched, the method asks which scientifically meaningful direction of misspecification is supported by the data. The amortized inference strategy is also computationally appealing.

Critical weaknesses

The reviewer's main concern is that the inferred discrepancy posterior relies on several approximations, including approximate posterior sampling and ELBO-based density ratios. The Gaussian example validates these approximations in a controlled setting, but there is no direct calibration against a non-amortized evidence computation on the realistic experiments.

The M87* conclusion also appears somewhat stronger than what the experiment alone establishes. A high discrepancy relative to simulated GRMHD observations indicates mismatch along the proposed compact-emission direction, but other forms of prior misspecification could potentially be absorbed by the same discrepancy family.

The related-work discussion could also be potentially enriched by connecting PIVOT to recent work [1-7] on inference/test-time adaptation and scaling with diffusion priors, which can all be thought of as ensemble-based posterior inference methods. Therefore, the authors are strongly encouraged to expand the discussion of related work on inference/test-time methods with diffusion-based priors, which helps enhance quality of the manuscript.

Questions for authors
  1. Would it be possible for the authors to compare the amortized p(a∣y)p(a\mid y) against a non-amortized reference obtained by solving the inverse problem separately for several fixed values of aa?

  2. How robust is the M87* compact-emission conclusion to alternative discrepancy families? Could another generic prior mismatch also lead to a large inferred emission discrepancy?

References:

[1] Singhal, R., Horvitz, Z., Teehan, R., Ren, M., Yu, Z., McKeown, K., & Ranganath, R. (2025). A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.06848.

[2] Skreta, M., Akhound-Sadegh, T., Ohanesian, V., Bondesan, R., Aspuru-Guzik, A., Doucet, A., ... & Neklyudov, K. (2025). Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503.02819.

[3] Uehara, M., Zhao, Y., Wang, C., Li, X., Regev, A., Levine, S., & Biancalani, T. (2025). Inference-time alignment in diffusion models with reward-guided generation: Tutorial and review. arXiv preprint arXiv:2501.09685.

[4] He, J., Hernández-Lobato, J. M., Du, Y., & Vargas, F. (2025). Rne: a plug-and-play framework for diffusion density estimation and inference-time control. arXiv e-prints, arXiv-2506.

[5] Wang, C., Wang, W., Ren, Y., Blanchet, J., & Lu, Y. (2026, May). Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures. In Forty-third International Conference on Machine Learning.

[6] Ren, Y., Gao, W., Ying, L., Rotskoff, G., & Han, J. (2026, April). Driftlite: Lightweight drift control for inference-time scaling of diffusion models. In International Conference on Learning Representations (Vol. 2026, pp. 76664-76702).

[7] Chen, H., Ren, Y., Min, M. R., Ying, L., & Izzo, Z. (2025). Solving inverse problems via diffusion-based priors: An approximation-free ensemble sampling approach. arXiv preprint arXiv:2506.03979.

3 comments
AuthorsOpenReview ↗

Author Response to 1Wdz (1)

We thank the reviewer for recognizing our work as “intuitive and useful” and “computationally appealing”. We address their concerns below.

W1, Q1\textbf{W1, Q1}. For the ELBO concern, see W1\textbf{W1} response for Reviewer Np1P where we extensively validate the ELBO approximation and find that it matches both the highly discretized, integrated PF-ODE density and recovers the generating ground truth condition.

Sampling p(x∣y,a)p(x\mid y,a) for a fixed aa does not directly provide the evidence p(y∣a)p(y \mid a) which is required for constructing a non-amortized reference p(a∣y)p(a \mid y), and obtaining reliable evidence estimates for diffusion priors is difficult. Existing evidence estimators such as DiME [1] have not been extended to the posterior sampling method we use in our paper and also demonstrate that classical estimators lead to biased results on diffusion priors. Thus, the suggested experiment would require significant extension and validation of a new estimator. To address the concern, we run experiments testing the amortization assumption itself: whether the amortized head p(x∣∅)p(x \mid \varnothing) represents the uniform mixture ∫p(x∣a)p(a)da\int p(x \mid a) p(a) da as suggested by reviewer 5LP2. If this holds, the amortized pamort(a∣y)p_{amort} (a \mid y) is mathematically identical to the non-amortized pnonamort(a∣y)p_{nonamort} (a \mid y); otherwise, it tells us how much amortization error can distort a∣ya \mid y:

\propto p_{\mathrm{nonamort}}(a\mid y) \mathbb{E}_{x\sim p(x\mid y,a)} \left[ \frac{p(x\mid\varnothing)} {\int p(x\mid a')p(a')\\,da'} \right].$$ See $\textbf{W2}$ response for Np1P for results: $p(x \mid \varnothing)$ closely matches $\int p(x \mid a) p(a) da$ in both image-space FID up to finite-sample variability as well as condition-space $W_1$, providing empirical support that the amortization is valid. Finally, the synthetic Gaussian experiments include the effects of the approximate posterior sampling method, yet still closely match the analytic evidence. We will add these results to the final version of the paper. [1] Wang et al. Sample-efficient evidence estimation of score based priors for model selection, ICLR 2026. $\textbf{W2, Q2}$. We agree that the emission family is not the unique explanation; our method tests whether a specific physically motivated hypothesis is more strongly supported by the data via the Bayesian evidence, in which case the M87* observations result in greater support for compact emission discrepancies than in-distribution GRMHD. For alternate discrepancies, we already demonstrate that elliptically perturbed measurements lead to similarly low emission discrepancy. While this does not account for every possible discrepancy, it does confirm that a high emission discrepancy does not always occur from GRMHD prior mismatch. Many discrepancies, such as variations in ring diameter (35uas - 50uas), spin (-0.94, -0.5, 0, 0.5, 0.94), or inclination (0 to 180 deg) are already captured in the ordinary variability of our GRMHD training dataset. On the other hand, additional ellipticity and off-ring emission are two plausible, physically motivated, and actively studied [2-3] open questions that are not captured by our GRMHD simulations, which is why they were specifically chosen. Therefore, while another mismatch could lead to a large inferred emission discrepancy, these two discrepancies are particularly of interest for M87*. [2] Dahale et al. Origin of the ring ellipticity in the black hole images of M87∗, A&A 2025 [3] Muller et al. Probing jet base emission of M87* with the 2021 Event Horizon Telescope observations, A&A 2026 $\textbf{W3}$. We thank the reviewer for pointing out these references, and we will expand upon the discussion of diffusion posterior inference in the final version.
Reviewer_1WdzOpenReview ↗

Response

The reviewer would like to thank the authors for providing a detailed response and uploading the revised manuscript, which has satisfactorily addressed most of the concerns raised in the original review. Consequently, the reviewer would be glad to raise the rating from 2 to 3.

AuthorsOpenReview ↗

Comment by Authors

Thank you for the thoughtful response and considering our revised manuscript!

Reviewer_5LP2OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PIVOT investigates specific ways a simulation-trained diffusion prior may be misspecified. It trains a conditional diffusion model over a user-defined family of perturbations, then reuses samples from the marginalized image posterior to estimate p(a∣y)p(a\mid y) through ELBO-based density ratios. The inferred perturbation strength is compared with synthetic reference observations. Validation includes an analytic Gaussian experiment, travel-time tomography, and M87* observations. The M87* results suggest additional compact emission under the chosen model, while ellipticity is weakly constrained.

Critical strengths
  1. PIVOT asks which specified changes to a prior the observations support, rather than only reporting distribution shift. The marginalization argument in Eqs. 8–15 is sound under the stated assumptions, and reusing one set of posterior samples avoids solving the inverse problem separately for every perturbation strength. This makes the framework useful for examining scientific assumptions. its an important question with an interpretable and practical method. 2 It explicitly illustrates invisible or misleading perturbations, and the experiments include weakly constrained cases rather than only successful detections. Comparing real observations with synthetic references is also helpful for distinguishing an unusual result from ordinary uncertainty.
Critical weaknesses
  1. The reliability of the estimated probabilities needs stronger validation. The Gaussian experiment uses an analytic score, so it does not test learned-model errors. Appendix A.4 provides useful recovery examples, but does not separate limited measurement sensitivity from errors introduced by the learned model and ELBO approximation. In particular, upward estimates of weak ellipticity need not imply incorrect inference, but their cause needs clearer diagnosis.
  2. Support for the chosen perturbation family does not uniquely establish the proposed physical explanation. The CurveFaultB example illustrates this: the chamber family can explain structural mismatches even when the ground truth comes from outside that family.
  3. For M87*, the consistent emission location is encouraging, but sensitivity to the strength and location priors, and the frequency of similarly consistent false detections, are not quantified. These checks matter for distinguishing physical emission from other model mismatches.
  4. The ellipticity conclusion needs narrowing: weak sensitivity supports <insufficient evidence for additional ellipticity> not the stronger claim that the true ellipticity lies within GRMHD variability.
Questions for authors
  1. What are the inferred component’s angular offset and flux, and which measurements constrain it?
  2. Does averaging the estimated p(a∣x)p(a\mid x) over samples from the marginalized prior recover the uniform training distribution?
  3. Can you report a Bayes factor comparing a=0 with an explicitly defined emission alternative?
3 comments
AuthorsOpenReview ↗

Author Response to 5LP2 (1)

We thank the reviewer for recognizing our work as “sound under the stated assumptions” and addressing “an important question with an interpretable and practical method”. We address their concerns below.

W1\textbf{W1}. See W1\textbf{W1} response for Reviewer Np1P where we extensively validate the ELBO approximation (including the ellipticity case) and find that it matches both the highly discretized, integrated PF-ODE density and generating ground truth condition closely. Since we also validate the other approximations (see W1\textbf{W1} response for 1Wdz) and the elliptic discrepancy posterior tends to be broad, this suggests that the ellipticity is not constrained by the limited measurements. We will add this discussion to the final version of the paper.

W2\textbf{W2}, W3\textbf{W3}. We agree that this is an important problem and we use CurveFaultB to demonstrate a case where a non-unique discrepancy is detected, but our method tests whether a specific physically motivated hypothesis is supported by the data via the Bayesian evidence rather than necessarily being the unique physical explanation. Furthermore, the posterior samples of the chamber-perturbed CurveFaultB still capture a significant feature of the truth and result in more accurate reconstructions compared to using the base FlatVelA prior.

While p(a∣y)p(a \mid y) depends on the prior, the discrepancy family spans a range of physically motivated and plausible strengths and locations, so our conclusions already account for variability within this family. We also test the false emission detections in the paper: we compute the discrepancy posterior of 64 in-distribution GRMHD measurements and find that only 2 of them result in high emission discrepancy (Figure 5) on April 5 and 6, but the emission has inconsistent location and is consistent with posterior uncertainty rather than a consistent localized feature under both component fitting and visual inspection (Appendix A.3). Therefore, of the 64 in distribution measurements, none of them have a consistent off-ring emission like M87*.

W4\textbf{W4}. Thank you for the point, we will narrow the claim in the final version of the paper.

Q1\textbf{Q1}. We report angular offset and flux below. The ALMA-LMT-SMT triangle of telescopes showed the greatest sensitivity under a re-observation of the reconstructed image containing the component.

DayAngular offset (μ\muas)Flux (mJy)
April 556.9±2.756.9 \pm 2.723.6±16.923.6 \pm 16.9
April 655.0±3.455.0 \pm 3.418.5±12.418.5 \pm 12.4
April 1056.3±2.956.3 \pm 2.919.5±15.919.5 \pm 15.9
April 1156.1±3.556.1 \pm 3.58.6±5.18.6 \pm 5.1

Q2\textbf{Q2}. Yes; see W2\textbf{W2} response for Reviewer Np1P.

Q3\textbf{Q3}. We can define our Bayes factor as p(y∣a)/p(y∣a=0)=p(a∣y)/p(a=0∣y)p(y \mid a) / p(y \mid a=0) = p(a \mid y) / p(a=0 \mid y) for different emission strengths aa corresponding to varying flux density FF that are chosen a priori. The results indicate that with these fixed emission hypotheses, there is moderate support for 17.25 mJy flux emission on April 5 and weak support for 8.62 mJy flux emission on April 10.

Daya=0.1a=0.1a=0.2a=0.2a=0.3a=0.3a=0.4a=0.4a=0.5a=0.5
FF (mJy)8.6217.2525.8734.4943.12
April 51.775.760.9561.191.26
April 60.9221.500.7290.3880.143
April 102.571.970.4940.5510.338
April 111.440.4710.01411.79e-62.09e-6
Reviewer_5LP2OpenReview ↗

Response to author

thanks for your reply! The rebuttal addresses most of my concerns. However, the calibration of the ellipticity estimates remains uncertain, and the claim of additional emission in M87* should be more cautious. Consequently, I am inclined to raise my rating to 3.

AuthorsOpenReview ↗

Comment by Authors

Thank you for the thoughtful response! Given the remaining concern about ellipticity calibration, we have already updated our conclusions about ellipticity in the revised paper PDF to reflect inconclusivity due to the uncertain posteriors and high inferred discrepancies even for in-distribution GRMHD. Additionally, our claim about additional emission in M87* already reflects a cautious stance: "This emission is one explanation that reduces the disagreement between the M87* observations and the GRMHD prior".

Reviewer_Np1POpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PIVOT, a framework for probing diffusion-prior misspecification through scientifically motivated perturbation families. A conditional diffusion model represents priors indexed by discrepancy strength, while label dropout learns the marginalized prior. Posterior samples under this marginalized prior are reused to estimate the discrepancy posterior through ELBO-based conditional density ratios. Experiments cover an analytic Gaussian problem, seismic tomography, and real M87 observations.

Critical strengths
  1. The method asks which specified departures from a learned prior are supported by observations, providing more actionable information than a generic OOD score.
  2. Under the stated hierarchical model, Eqs. 9–15 correctly express the discrepancy posterior using samples from the marginalized image posterior, avoiding separate inverse-problem solves at every discrepancy value.
  3. Figure 1 and the ellipticity experiments acknowledge that observational agreement cannot resolve arbitrary null-space changes. The analytic experiment and noise-sensitivity studies are useful checks.
Critical weaknesses
  1. ELBO-based density ratios are not sufficiently validated for the learned priors. The condition-independent term C(x) cancels in the ratio, but the remaining ELBO gap can depend on discrepancy strength and therefore distort posterior probabilities.
  2. The learned unconditional and conditional distributions may not satisfy the exact mixture identity (Eqs. 7). Image posterior sampling uses the dropout-conditioned model, whereas discrepancy estimation normalizes densities from the conditional model. With finite training, these two representations need not correspond to exactly the same mixture distribution. This introduces an additional approximation beyond ELBO evaluation and finite sampling, but its effect on the inferred discrepancy posterior is not assessed.
  3. Compact emission is restricted to a particular radial range, shape distribution, and amplitude range, with a uniform hyperprior on strength. These choices affect the evidence through marginalization over nuisance variables. The paper acknowledges dependence on the discrepancy family, but does not demonstrate robustness to reasonable alternative parameterizations or hyperpriors. Separate emission and ellipticity analyses also do not resolve their possible joint confounding.
Questions for authors

see weaknesses.

2 comments
AuthorsOpenReview ↗

Author Response to Np1P (1)

We thank the reviewer for recognizing our work as “providing more actionable information than a generic OOD score”. We address their concerns here:

W1\textbf{W1}. We provide validation that the ELBO approximation recovers the correct p(a∣y)p(a \mid y) shape in the synthetic Gaussian cases in Section 4.1 and Appendix A.2; while the score is analytic and exact in this setting, the ELBO gap still exists. We also demonstrate that the ELBO-based p(a∣y)p(a \mid y) assigns a high density to the ground truth discrepancy condition under the learned score in Appendix A.4, where both the learned score and the approximate posterior sampling are used. Previous works have also validated the ELBO for recovering dark matter simulator conditions [1], classifying images [2], and variational inference in place of expensive PF-ODE integration [3-4].

Furthermore, we use the ELBO for efficiency and differentiability, allowing for MCMC sampling; for a low-dimensional grid-based evaluation of p(a∣y)p(a \mid y), we can evaluate the exact density (up to discretization error) by integrating the Jacobian along the PF-ODE [5], providing a way to quantify ELBO approximation errors. We provide a comparison between the ELBO and the PF-ODE integrated density for 100 samples of the two GRMHD models below, across a range of PF-ODE discretization steps, and report the following results: (1) Mean error\textbf{Mean error} to ground truth label ∥E[a∣x]−a∗∥\|E[a\mid x] - a^* \|, (2) W1 distanceW_1 \textbf{ distance} to the highly discretized PF-ODE reference W1(p(a∣x),ppf-ode 400(a∣x)))W_1( p(a\mid x), p_{\text{pf-ode 400}}(a\mid x))) and (3) computational efficiency.

GRMHD compact-emission discrepancy model

MethodMean error ↓\downarrowW1W_1 to reference ↓\downarrowTime (s) ↓\downarrowRelative cost
ELBO (1 pair)0.0134±0.00990.0134 \pm 0.00990.0058±0.00700.0058 \pm 0.00701.041.00×1.00\times
PF-ODE (25 steps)0.1177±0.06720.1177 \pm 0.06720.1163±0.06520.1163 \pm 0.06521.691.62×1.62\times
PF-ODE (50 steps)0.0246±0.02130.0246 \pm 0.02130.0203±0.02000.0203 \pm 0.02003.373.23×3.23\times
PF-ODE (100 steps)0.0146±0.01080.0146 \pm 0.01080.0075±0.00880.0075 \pm 0.00886.736.46×6.46\times
PF-ODE (200 steps)0.0149±0.01020.0149 \pm 0.01020.0041±0.00550.0041 \pm 0.005513.4812.93×12.93\times
PF-ODE (400 steps)0.0140±0.00970.0140 \pm 0.00970.0000±0.00000.0000 \pm 0.000027.2426.14×26.14\times

GRMHD ellipticity discrepancy model

MethodMean error ↓\downarrowW1W_1 to reference ↓\downarrowTime (s) ↓\downarrowRelative cost
ELBO (1 pair)0.0430±0.03180.0430 \pm 0.03180.0092±0.00820.0092 \pm 0.00821.041.00×1.00\times
PF-ODE (25 steps)0.1209±0.10790.1209 \pm 0.10790.1001±0.09580.1001 \pm 0.09581.691.62×1.62\times
PF-ODE (50 steps)0.0558±0.04180.0558 \pm 0.04180.0246±0.01990.0246 \pm 0.01993.383.23×3.23\times
PF-ODE (100 steps)0.0448±0.03410.0448 \pm 0.03410.0101±0.00790.0101 \pm 0.00796.746.45×6.45\times
PF-ODE (200 steps)0.0422±0.03340.0422 \pm 0.03340.0049±0.00380.0049 \pm 0.003813.5212.94×12.94\times
PF-ODE (400 steps)0.0414±0.03310.0414 \pm 0.03310.0000±0.00000.0000 \pm 0.000027.3426.17×26.17\times

As can be seen, the ELBO agrees with the highly discretized PF-ODE reference at significantly lower cost, and both give comparable mean error metrics; note that the non-zero MAE is because the ground truth condition does not necessarily need to be the posterior mean. We will describe this option and compare tradeoffs in accuracy and computation in the final version of this paper.

[1] Mudur et al. Diffusion-HMC: Parameter Inference with Diffusion-model-driven Hamiltonian Monte Carlo, ApJ 2025.

[2] Li et al. Your diffusion model is secretly a zero-shot classifier, ICCV 2023.

[3] Feng et al. Score-Based Diffusion Models as Principled Priors for Inverse Imaging, ICCV 2023.

[4] Feng et al. Variational Bayesian Imaging with an Efficient Surrogate Score-based Prior, TMLR 2024.

[5] Song et al. Maximum Likelihood Training of Score-Based Diffusion Models, NeurIPS 2021.

AuthorsOpenReview ↗

Author Response to Np1P (2)

W2\textbf{W2}. We validate the amortized head in both (1) condition-space and (2) image-space. For (1), we evaluate the distribution p(a∣x)p(a \mid x) of 500 unconditional samples of p(x∣∅)p(x \mid \varnothing) using the ELBO which we validated in W1\textbf{W1}; if the mixture is correctly represented then we have Ex∼p(x∣∅)[p(a∣x)]=∫p(a∣x)p(x∣∅)dx=p(a)=Unif[0,1]\mathbb{E}_{x \sim p(x \mid \varnothing)} [p(a \mid x)] = \int p(a \mid x) p(x \mid \varnothing) dx = p(a) = Unif[0,1], the uniform distribution. The W1W_1 distance between them is reported below.

PriorW1(Ex∼p(x∣∅)[p(a∣x)],Unif[0,1])W_1(\mathbb{E}_{x \sim p(x \mid \varnothing)}[p(a \mid x)],\mathrm{Unif}[0,1])
GRMHD + Ellipticity0.0118
GRMHD + Compact Emission0.0085
FlatVelA + Chambers0.0096

For (2), we compare 2000 samples of p(x∣∅)p(x \mid \varnothing) with 2000 samples generated by drawing a∼p(a)a \sim p(a) and then x∼p(x∣a)x \sim p(x \mid a). We consider (1) uniform p(a)p(a) (which is the ideal mixture distribution) and (2) truncated exponential and beta distributed p(a)p(a) which are biased. We also compare FID with an independently drawn set of 2000 samples of p(x∣∅)p(x \mid \varnothing) to determine the effect of finite sample bias.

ComparisonFID
p(x∣∅)↔p(x∣∅)p(x \mid \varnothing) \leftrightarrow p(x \mid \varnothing)3.323
p(x∣∅)↔a∼Unif[0,1],;x∼p(x∣a)p(x \mid \varnothing) \leftrightarrow \\{a \sim \mathrm{Unif}[0,1],\\; x \sim p(x \mid a)\\}3.361
p(x∣∅)↔a∼Exp(3),;x∼p(x∣a)p(x \mid \varnothing) \leftrightarrow \\{a \sim \mathrm{Exp}(3),\\; x \sim p(x \mid a)\\}14.015
p(x∣∅)↔a∼Beta(2,1),;x∼p(x∣a)p(x \mid \varnothing) \leftrightarrow \\{a \sim \mathrm{Beta}(2,1),\\; x \sim p(x \mid a)\\}7.086
a∼Unif[0,1],;x∼p(x∣a)↔a∼Exp(3),;x∼p(x∣a)\\{a \sim \mathrm{Unif}[0,1],\\; x \sim p(x \mid a)\\} \leftrightarrow \\{a \sim \mathrm{Exp}(3),\\; x \sim p(x \mid a)\\}12.060
a∼Unif[0,1],;x∼p(x∣a)↔a∼Beta(2,1),;x∼p(x∣a)\\{a \sim \mathrm{Unif}[0,1],\\; x \sim p(x \mid a)\\} \leftrightarrow \\{a \sim \mathrm{Beta}(2,1),\\; x \sim p(x \mid a)\\}8.542
a∼Exp(3),;x∼p(x∣a)↔a∼Beta(2,1),;x∼p(x∣a)\\{a \sim \mathrm{Exp}(3),\\; x \sim p(x \mid a)\\} \leftrightarrow \\{a \sim \mathrm{Beta}(2,1),\\; x \sim p(x \mid a)\\}29.202

The amortized condition distribution closely matches the uniform distribution, and the image distribution agrees with the true mixture distribution (up to finite sample bias, as demonstrated by the FID between p(x∣∅)p(x \mid \varnothing) and itself). We will add all of these results to the paper.

W3\textbf{W3}. While p(a∣y)p(a \mid y) depends on the prior, the discrepancy family spans a range of physically motivated and plausible strengths and locations, so our conclusions already account for variability within this family. A hyperprior also does not alter the underlying evidence p(y∣a)p(y \mid a) of each measurement; therefore the gap in evidence between in distribution and OOD measurements is not affected for any discrepancy including compact emission. Meanwhile, changing the radial range, shape distribution, and amplitude range constitutes a different hypothesis and could result in different evidence. Importantly, our method tests whether a specific physically motivated hypothesis is supported by the data via the Bayesian evidence rather than necessarily being the unique physical explanation.

As for dependence or confounding of discrepancy families, we agree that this is an important problem and discuss it in great detail, but this is a limitation arising from ill-posedness of limited measurements. For alternate discrepancies, we already demonstrate that elliptically perturbed measurements lead to similarly low emission discrepancy. While this does not account for every possible discrepancy, it does confirm that a high emission discrepancy does not always occur from GRMHD prior mismatch. Additionally, many discrepancies, such as variations in ring diameter (35uas - 50uas), spin (-0.94, -0.5, 0, 0.5, 0.94), or inclination (0 to 180 deg) are already captured in the ordinary variability of our GRMHD training dataset. On the other hand, additional ellipticity and off-ring emission are two plausible, physically motivated, and actively studied [6-7] open questions that are not captured by our GRMHD simulations, which is why they were specifically chosen.

[6] Dahale et al. Origin of the ring ellipticity in the black hole images of M87∗, A&A 2025

[7] Muller et al. Probing jet base emission of M87* with the 2021 Event Horizon Telescope observations, A&A 2026

Comments on the paper

2 comments
AuthorsOpenReview ↗

Message to AC

We thank all of the reviewers for their constructive feedback and look forward to the AC’s attention on the matter. The reviewers’ concerns can be summarized by three main points: (1) insufficient validation of ELBO and posterior sampling method, (2) the unconditional head not necessarily representing the true uniform mixture and (3) possible confounding, non-uniqueness, or sensitivity to the discrepancy family. The first two can be directly addressed through a small set of new experiments, which we describe below and will add to the final version of the paper. We acknowledge (3) throughout the work as well as provide calibration methods to assess if discrepancies can be identified from our measurements as well as detecting false positives and negatives.

(1) Many reviewers brought up possible errors as a result of using the ELBO and approximate posterior sampling. The ELBO recovers p(a∣y)p(a \mid y) in the synthetic Gaussian cases in Section 4.1 and Appendix A.2; while the score is analytic and exact in this setting, the ELBO gap still exists. These experiments also use the same posterior sampling approach, thereby jointly testing the ELBO and approximate posterior sampler; however, we emphasize that PIVOT’s methodology is agnostic to the particular diffusion model and posterior sampling method. We also demonstrate that the ELBO-based p(a∣y)p(a \mid y) assigns a high density to the ground truth discrepancy condition under the learned score in Appendix A.4, where both the learned score and the approximate posterior sampling are used. Previous works have also validated the ELBO for recovering dark matter simulator conditions [1], classifying images [2], and variational inference in place of expensive PF-ODE integration [3-4].

We can also evaluate the learned diffusion density (up to discretization error) by integrating the Jacobian trace along the PF-ODE [5], providing a way to quantify ELBO approximation errors. We provide a comparison between the ELBO and the PF-ODE integrated density; see W1\textbf{W1} response to Np1P. The ELBO agrees with the highly discretized PF-ODE reference at significantly lower cost, and comparable mean error metrics.

(2) Reviewers also mentioned concerns over the amortized unconditional head, and whether or not it can represent a uniform mixture of conditions under finite model capacity. We validate the amortized head in both (1) condition-space and (2) image-space in W2\textbf{W2} response to Np1P, where we find that the amortized condition distribution closely matches the uniform distribution and the FID between the unconditional amortized distribution and the true mixture distribution is comparable to the finite sample FID bias.

(3) Reviewers were also concerned about the non-uniqueness of the discrepancy family, possible confounding with other discrepancy effects and sensitivity to the chosen prior. While the evidence depends on the prior, our proposed discrepancy families span a range of physically motivated and plausible strengths and locations, so our conclusions already account for variability within this family. Meanwhile, changing the radial range, shape distribution, and amplitude range constitutes a different hypothesis and could result in different evidence. We emphasize that our method tests whether a specific, scientifically motivated hypothesis is supported by the data via the Bayesian evidence rather than necessarily being the unique physical discrepancy.

For the black hole experiment, many discrepancies, such as variations in ring diameter, spin, or inclination are already captured in the ordinary variability of our GRMHD training dataset. On the other hand, additional ellipticity and off-ring emission are two plausible, physically motivated, and actively studied [6-7] open questions that are not captured by our GRMHD simulations, which is why they were specifically chosen. Therefore, while another mismatch could lead to a large inferred emission discrepancy, these two discrepancies are particularly of interest for M87*.

[1] Mudur et al. Diffusion-HMC: Parameter Inference with Diffusion-model-driven Hamiltonian Monte Carlo, ApJ 2025.

[2] Li et al. Your diffusion model is secretly a zero-shot classifier, ICCV 2023.

[3] Feng et al. Score-Based Diffusion Models as Principled Priors for Inverse Imaging, ICCV 2023.

[4] Feng et al. Variational Bayesian Imaging with an Efficient Surrogate Score-based Prior, TMLR 2024.

[5] Song et al. Maximum Likelihood Training of Score-Based Diffusion Models, NeurIPS 2021.

[6] Dahale et al. Origin of the ring ellipticity in the black hole images of M87∗, A&A 2025

[7] Muller et al. Probing jet base emission of M87* with the 2021 Event Horizon Telescope observations, A&A 2026

AuthorsOpenReview ↗

Updated paper PDF

We have just uploaded a revised version of the paper PDF with all of the changes and additions described below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.