acceptodds
Under review as a conference paper at ICLR 2027

Is your uncertainty map wrong, or is its target? Exact diagnostics for the Tweedie diagonal, and a gradient-free alternative

Abstract

Diffusion models can predict a follow-up medical scan from a baseline, and a clinician reading such a prediction needs a per-voxel map of where it should be trusted. Many of these maps approximate the diagonal of the Tweedie posterior covariance, and they are evaluated by comparing one approximation against another, so it remains unclear whether the estimator or the target is the limiting factor. We therefore compute the exact diagonal on six checkpoints across fourteen model–corpus conditions. A Hutchinson estimator at M=200 tracks it at rank agreement of at least 0.92 in every condition. However, in four of the fourteen the exact diagonal is anti-correlated with the denoising error, reaching −0.13, so a faithful estimator reproduces that reversal. All four are real-image conditions, and in these checkpoints the reversal does not appear on the models' own samples, so evaluation on model-generated samples gives a favourable reading of this family. From this one may conclude that the limiting factor is the target rather than the estimator. We then introduce Tweedie Probe-Tangent (T-PT), a gradient-free residual probe that corrupts one model-supported prediction repeatedly and measures the voxel-wise variance of the denoiser's response. T-PT reads a different functional of the same Jacobian, and its exact second-order form ranks identically with the diagonal wherever the diagonal reverses. At the working budget of thirty probes, however, T-PT returns a map too unstable to reproduce that ranking, whilst a Hutchinson estimator at M=5 is already stable enough to reproduce it; T-PT at that budget is therefore too unstable to be read as evidence against the reversal. Where the diagonal is not reversed, T-PT tracks the diagonal closely in most conditions, at a small fraction of the cost of the exact computation. We therefore offer T-PT as an instrument rather than as a better approximation, and we evaluate it on clinical follow-up prediction from a single reverse chain, on brain magnetic resonance imaging (MRI) and on lung computed tomography (CT). On brain MRI at full resolution, where every Jacobian-based estimator we test exhausts memory, T-PT leads the twenty-chain Monte-Carlo ensemble on five of the eight endpoints measured inside tissue and trails it on none, at 16× fewer network evaluations. Over the whole volume the ensemble leads instead, on a comparison carried by the background that fills three quarters of it, and a fifty-chain ensemble closes the tissue gap on a thirty-volume subset. On lung CT the ensemble is ahead on every endpoint, whilst a Hutchinson estimate of that diagonal ranks the error well below T-PT. Both estimators lose most of their discrimination in the regions of longitudinal change, which remains an open problem.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
diffusion models, uncertainty quantification, Tweedie posterior covariance, Jacobian diagonal estimation, Hutchinson estimator, per-voxel uncertainty maps, medical image synthesis, longitudinal prediction, follow-up scan prediction, brain MRI, lung CT, Monte-Carlo DDIM, variance propagation, error ranking, evaluation methodology

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_y3TuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates a fundamental question in diffusion-model uncertainty estimation: when an uncertainty map fails to identify erroneous predictions, is the problem caused by an inaccurate estimator, or by the underlying uncertainty quantity being estimated?

The central empirical observation is interesting. Although a Hutchinson estimator with 200 probes achieves at least 0.92 rank agreement with the exact diagonal in every tested condition, the exact diagonal is negatively correlated with denoising error in four conditions, reaching approximately −0.13.

The paper is also unusually candid about its proposed method's limitations: T-PT does not reliably resolve its own target in these negative-correlation conditions at a budget of 30 probes, and higher probe counts can recover the same undesirable ranking.

Critical strengths

The paper makes a valuable conceptual distinction between estimator error and target failure. It provides exact Jacobian-derived reference calculations, a mathematically motivated gradient-free estimator, extensive controls, and thoughtful statistical reporting. Particularly commendable is the explicit acknowledgment that T-PT's apparently neutral performance in the negative-correlation conditions can result from insufficient estimator reliability.

Critical weaknesses

Section 3 computes the Jacobian-derived quantity implied by a learned denoiser. Negative diagonal entries are subsequently observed and clamped. This means the computed object is exact with respect to the fitted network's Jacobian, but it is not necessarily the true Bayesian posterior covariance.

The four negative-correlation conditions in Appendix Table 7 involve two unconditional pixel-space checkpoints, each evaluated on two real-image corpora. Although the paper reports this correctly, the repeated emphasis on “four of fourteen” may suggest broader architectural diversity than the data demonstrate.

Figure 3 (page 33) includes useful brain MRI and lung CT overlays, but the anatomical images and uncertainty overlays are small. Each uncertainty map is normalized to its own quantiles, making relative magnitude comparisons impossible. For a paper concerned with spatial error localization, I recommend enlarging the figure and adding a second example focused on failure regions. For example, a zoomed lung nodule boundary could show where MC-DDIM identifies large errors that T-PT fails to prioritize. A brain MRI example centered on longitudinal anatomical change would be similarly useful.

The paper nonetheless combines three substantial contributions: exact Tweedie-diagonal diagnostics, geometric analysis of residual probing, and clinical evaluation of a computationally cheaper uncertainty estimator. These components are related but do not establish one another automatically. For example, the natural-image experiments identify negative error correlations, whereas the clinical experiments demonstrate a performance–cost trade-off without determining whether the exact Tweedie diagonal would exhibit the same reversal on the clinical checkpoints. The manuscript does acknowledge this boundary in Section 6, but it should be emphasized earlier. The theoretical analysis of analytical variance-propagation instability is useful, though somewhat secondary to the central diagnostic contribution.

T-PT itself is closely related to existing perturbation-based uncertainty estimation and test-time augmentation. Section 2 acknowledges this connection, particularly with De Vita and Belagiannis (2025). The residual construction is mathematically meaningful, but the broader strategy of repeatedly perturbing a prediction and measuring output variability is not fundamentally new. Furthermore, T-PT is not shown to resolve the negative-ranking problem identified in the first part of the paper. Appendix Tables 11 and 12 establish that its second-order functional can retain the same undesirable ranking, while finite-budget estimation may simply be too unstable to reproduce it. The authors acknowledge this limitation, which strengthens the scientific credibility of the work but narrows the innovative claim.

Questions for authors

Section 3.3; Appendix Table 7. The main diagnostic evidence is strongest for two unconditional diffusion models evaluated on real images. The study includes a medical diffusion checkpoint, but the exact evaluation there uses model-generated two-dimensional latent samples rather than real longitudinal clinical observations. Consequently, the clinical experiments do not establish that the exact diagonal is negatively correlated with reconstruction error in the actual medical forecasting setting.

Sections 4.4 and 5.1; Appendix Tables 12, 18, and 23. This is an important methodological concern. The paper demonstrates that T-PT at K=30K=30 can produce unreliable maps in natural-image conditions where the underlying signal is weak. However, the main brain MRI experiment uses only K=15K=15 probes. Appendix Table 18 reruns experiments with alternative probe-chain seeds and shows stable performance contrasts. This is useful, but stable aggregate correlations do not establish that two independently generated uncertainty maps give similar rankings to individual voxels.

Section 6 and Appendix P. Both methods perform poorly in regions of longitudinal change. This is not a minor limitation: these are precisely the locations where a medical follow-up prediction is likely to be clinically informative. The paper appropriately admits that this remains unresolved, but the finding should receive greater prominence in the practical conclusions.

Section 6; Appendix Tables 20–21. The authors conduct a valuable fifty-chain experiment. It shows that the twenty-chain ensemble is still improving inside brain tissue. On the thirty-volume OASIS subset, T-PT's tissue advantage decreases from 0.038 to 0.010 when the reference expands from twenty to fifty chains. However, this subset contains only five participants and cannot support reliable population-level inference.

Appendix L; Appendix M; Appendix Table 19. The variance-head-free checkpoint is intended to demonstrate compatibility with a denoiser trained without a learned variance head. However, Appendix L states that this experiment uses the observed blur difference in the conditioning vector, whereas the principal deployment experiment zeroes the blur-difference component because it depends on information from both scans. Thus, the head-free experiment changes both the training objective and an important aspect of inference conditioning. Although the comparison demonstrates that T-PT can run without a variance head, it does not cleanly isolate the effect of removing that head on uncertainty-ranking performance.

Section 5.3; Appendices J and K. The paper reports that the Jacobian-based estimators exhaust memory on a 40 GB accelerator, while T-PT completes successfully. This is useful practical evidence, but Appendix K also explains that activation checkpointing was disabled to make forward-mode differentiation possible with the released implementations. Therefore, the observed memory failure is not a general impossibility result for Jacobian-based uncertainty estimation.

Reviewer_8jfuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1(−1)
Soundness
2
Read review
Paper summary

The present paper examines whether poor error ranking comes from estimation error or from the learned Tweedie covariance diagonal itself. Exact computations on six checkpoints across 14 conditions show that accurate diagonal estimation can still give negative correlations with denoising error. The authors propose T-PT, which repeatedly corrupts one model prediction and estimates the voxel-wise variance of the residual between injected and predicted noise, supported by a geometric analysis. In the clinical experiments, the 20-chain ensemble uses 13.4 to 16 times as many network evaluations as T-PT. T-PT shows modest gains inside brain tissue but lower ranking performance on lung CT. It does not resolve the diagnosed reversal.

Critical strengths

The exact-diagonal evaluation is the main strength of the paper. Hutchinson estimates with 200 probes have rank agreement of 0.92 to 0.996 with the exact learned diagonal, yet the exact diagonal is negatively correlated with denoising error in four of the 14 conditions. This distinguishes approximation error from a limitation of using the learned diagonal to rank denoising error. The diagnostic is useful because it identifies cases where more accurate approximation alone does not resolve poor error ranking.

Critical weaknesses
  1. Several details needed to interpret the central comparisons are unclear. In the exact-diagonal experiments, the authors average denoising error over 20 noise draws, but do not clearly specify which input is used for the Jacobian, whether the diagonal is also averaged, or how the T-PT anchor is selected for these comparisons. This makes these experiments difficult to reconstruct from the paper. Also, the clinical reference with 500 probes is called converged, but its stability across independent estimates at that budget or at larger budgets is not reported. The partial correlations show an association with error beyond this estimated diagonal, but the result for the exact clinical diagonal remains unclear. These issues limit confidence in the interpretation of the central comparisons.

  2. The practical advantage beyond reduced computation needs stronger support. The ensemble leads on all CT endpoints. Both methods also have much lower correlations with error in regions of actual change than across tissue overall. The MRI tissue advantage narrows with 50 chains, although this check includes only five participants. These acknowledged limitations restrict the demonstrated value for identifying errors in longitudinal change. Also, the memory comparison at 128 by 128 by 128 voxels uses automatic differentiation with checkpointing disabled for the tangent pass. A forward finite-difference Jacobian approximation, as described by Manor and Michaeli in 2024, could support diagonal estimation without this differentiation overhead, but its accuracy and feasibility here are not reported. This leaves unclear whether the memory advantage persists against a diagonal estimator using only forward passes.

Questions for authors
  1. In the exact-diagonal experiments, at which input is the Jacobian evaluated, and is the diagonal averaged over the 20 noise draws used for the error map? How is the T-PT anchor selected for each corpus?

  2. What evidence supports treating the clinical diagonal reference with 500 probes as converged? In particular, are the partial correlations stable across independent reference estimates or larger probe budgets?

  3. Was a diagonal estimator using forward finite-difference Jacobian approximations considered in the feasibility comparison at 128 by 128 by 128 voxels? Existing results or implementation constraints would help clarify this comparison. If no results are available, a targeted comparison of memory use and error-ranking accuracy would address the feasibility concern.

Reviewer_HUwmOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the pixel uncertainty map produced by diffusion denoisers. The authors identify the problem that some models can produce good covariance diagonal but but poor error ranking map. The authors propose a new way to compute the pixel uncertainty map, Tweedie Probe-Tangent (T-PT). T-PT work by generating a clean sample first, and then adding several independent noise to it, denoising and measuring the pixel variance of the residual between predicted and added noise. Experiments show that in some cases the proposed T-PT are better than MC-DDIM.

Critical strengths

The uncertainty map is important in practical clinical applications and is often overlooked. The paper works on a timely topic.

The method is practical as it requires only forward evaluations.

Critical weaknesses

The paper is quite difficult to read. The writing is way too AI-style, rhetorical, repetitive.

The authors acknowledge there are multiple methods that denoise, add noise and then estimate the variance of predicted noise. It seems for me the main difference between T-PT and the others, is that the authors compute the residual instead of noise. So my feeling is that the actual technical contribution is rather small.

The experiments and results are somewhat limited in the main paper. I feel like the important baseline from the "denoise-renoise" family are missing. Also it is not well ablated on the proposed T-PT itself, for example what is the impact of the renoise times to the final results. The computational aspects can benefit from having more results on actual clock time and memory etc.

Questions for authors

The authors call "exact diagonal", while the diagonal is actually of that specific learned diffusion denoiser. Only when the diffusion model so well trained that it really approximates the training distribution, I feel like it can be called a exact diagonal. So it is not the true posterior variance, but rather a posterior variance bounded to that specific denoiser?

The results show that the proposed T-PT can also produce incorrect ranking. So maybe the point here is more accessibilty than correctness?

Do the authors have potential explanations on why the method fails in changing regions?

Comments on the paper

1 comment

We thank the three reviewers. Two of them asked for measurements that the submission did not contain, and the third for results that it reported only in its appendices. We took advantage of this discussion period to make the former and to bring the latter into the main text. They are in the revised PDF and are summarised here for the area chair and for all reviewers.

New measurements

  1. Reliability of the clinical maps (y3Tu). Two maps built from independent probe sets agree at 0.944 [0.938, 0.949] inside tissue on lung CT (K=30) and at 0.808 [0.804, 0.812] on brain MRI (K=15), against at most 0.017 on the four reversed natural-image conditions (Table 25).
  2. A forward-only Jacobian baseline (8jfu). A Hutchinson estimate by finite differences runs at 1283128^3 at the memory of one forward pass, so the memory result of the submission belongs to forward-mode differentiation. At equal memory it ranks the error at about half the probe's value on both tasks: 0.333 against 0.671 inside lung tissue, and 0.195 against 0.388 inside brain tissue (App. X).
  3. Reliability of the M=500 reference (8jfu). Its two halves agree at 0.907 [0.882, 0.931], and "converged" is withdrawn (App. W).
  4. Definition of the error (8jfu). The negative correlations are against the error of the mean prediction, which is the aggregation the clinical evaluation scores; against two alternative definitions the same diagonals are near zero or weakly positive (Table 24).
  5. The nodule boundary (y3Tu). In a shell around the nodule the ensemble and the probe show no significant difference (paired difference −0.035-0.035 [−0.107-0.107, +0.032+0.032], twenty volumes); in the volume of Fig. 4 the probe's deficit in the change region lies in the vessels and the pleura.
  6. Controls and ablations (HUwm). The no-residual control, the sweeps over K and t*, and time and memory are now in the main text (Secs. 5.2 and 5.3).

Summary of changes in the revised PDF. Changed text is in blue. The numbering of the original sections, tables (1-23), appendices (A-U) and equations is unchanged; all new material is placed last (Apps. V-X, Tables 24-27) or in App. Q (Fig. 4).

Changes of substance

  1. Sec. 3: the denoising error is now defined correctly as the error of the mean prediction over twenty draws; Sec. 3.3 and Table 24 give the sensitivity to that definition, and Sec. 3 states that the diagonal is exact with respect to the learned denoiser.
  2. The word "converged" is withdrawn for the M=500 reference; App. W reports its split-half reliability.
  3. Sec. 5.2 now carries the no-residual control and the sweeps over K and t* in the main text.
  4. Sec. 5.3: the memory result is scoped to forward-mode differentiation; a finite-difference Hutchinson baseline is added (App. X, Tables 26-27, two rows in Table 5), with times per volume.
  5. Reliability of the clinical maps between independent probe sets (App. W, Table 25; Sec. 4.4).
  6. Sec. 6 and the conclusion state the regions of longitudinal change as the main practical limitation; a value from a different run was removed from Sec. 6.
  7. "Four of fourteen" is given with its composition, and the domain boundary is stated in Sec. 1.
  8. Fig. 3 is enlarged and Fig. 4 is added.
  9. App. L states that the head-free experiment does not isolate the effect of the head.

Changes of wording only. The remaining blue text renames headings and rewrites sentences for clarity, in the main text and the appendices. No number, reference or formula was changed by these edits.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.