- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates a fundamental question in diffusion-model uncertainty estimation: when an uncertainty map fails to identify erroneous predictions, is the problem caused by an inaccurate estimator, or by the underlying uncertainty quantity being estimated?
The central empirical observation is interesting. Although a Hutchinson estimator with 200 probes achieves at least 0.92 rank agreement with the exact diagonal in every tested condition, the exact diagonal is negatively correlated with denoising error in four conditions, reaching approximately −0.13.
The paper is also unusually candid about its proposed method's limitations: T-PT does not reliably resolve its own target in these negative-correlation conditions at a budget of 30 probes, and higher probe counts can recover the same undesirable ranking.
Critical strengths
The paper makes a valuable conceptual distinction between estimator error and target failure. It provides exact Jacobian-derived reference calculations, a mathematically motivated gradient-free estimator, extensive controls, and thoughtful statistical reporting. Particularly commendable is the explicit acknowledgment that T-PT's apparently neutral performance in the negative-correlation conditions can result from insufficient estimator reliability.
Critical weaknesses
Section 3 computes the Jacobian-derived quantity implied by a learned denoiser. Negative diagonal entries are subsequently observed and clamped. This means the computed object is exact with respect to the fitted network's Jacobian, but it is not necessarily the true Bayesian posterior covariance.
The four negative-correlation conditions in Appendix Table 7 involve two unconditional pixel-space checkpoints, each evaluated on two real-image corpora. Although the paper reports this correctly, the repeated emphasis on “four of fourteen” may suggest broader architectural diversity than the data demonstrate.
Figure 3 (page 33) includes useful brain MRI and lung CT overlays, but the anatomical images and uncertainty overlays are small. Each uncertainty map is normalized to its own quantiles, making relative magnitude comparisons impossible. For a paper concerned with spatial error localization, I recommend enlarging the figure and adding a second example focused on failure regions. For example, a zoomed lung nodule boundary could show where MC-DDIM identifies large errors that T-PT fails to prioritize. A brain MRI example centered on longitudinal anatomical change would be similarly useful.
The paper nonetheless combines three substantial contributions: exact Tweedie-diagonal diagnostics, geometric analysis of residual probing, and clinical evaluation of a computationally cheaper uncertainty estimator. These components are related but do not establish one another automatically. For example, the natural-image experiments identify negative error correlations, whereas the clinical experiments demonstrate a performance–cost trade-off without determining whether the exact Tweedie diagonal would exhibit the same reversal on the clinical checkpoints. The manuscript does acknowledge this boundary in Section 6, but it should be emphasized earlier. The theoretical analysis of analytical variance-propagation instability is useful, though somewhat secondary to the central diagnostic contribution.
T-PT itself is closely related to existing perturbation-based uncertainty estimation and test-time augmentation. Section 2 acknowledges this connection, particularly with De Vita and Belagiannis (2025). The residual construction is mathematically meaningful, but the broader strategy of repeatedly perturbing a prediction and measuring output variability is not fundamentally new. Furthermore, T-PT is not shown to resolve the negative-ranking problem identified in the first part of the paper. Appendix Tables 11 and 12 establish that its second-order functional can retain the same undesirable ranking, while finite-budget estimation may simply be too unstable to reproduce it. The authors acknowledge this limitation, which strengthens the scientific credibility of the work but narrows the innovative claim.
Questions for authors
Section 3.3; Appendix Table 7. The main diagnostic evidence is strongest for two unconditional diffusion models evaluated on real images. The study includes a medical diffusion checkpoint, but the exact evaluation there uses model-generated two-dimensional latent samples rather than real longitudinal clinical observations. Consequently, the clinical experiments do not establish that the exact diagonal is negatively correlated with reconstruction error in the actual medical forecasting setting.
Sections 4.4 and 5.1; Appendix Tables 12, 18, and 23. This is an important methodological concern. The paper demonstrates that T-PT at can produce unreliable maps in natural-image conditions where the underlying signal is weak. However, the main brain MRI experiment uses only probes. Appendix Table 18 reruns experiments with alternative probe-chain seeds and shows stable performance contrasts. This is useful, but stable aggregate correlations do not establish that two independently generated uncertainty maps give similar rankings to individual voxels.
Section 6 and Appendix P. Both methods perform poorly in regions of longitudinal change. This is not a minor limitation: these are precisely the locations where a medical follow-up prediction is likely to be clinically informative. The paper appropriately admits that this remains unresolved, but the finding should receive greater prominence in the practical conclusions.
Section 6; Appendix Tables 20–21. The authors conduct a valuable fifty-chain experiment. It shows that the twenty-chain ensemble is still improving inside brain tissue. On the thirty-volume OASIS subset, T-PT's tissue advantage decreases from 0.038 to 0.010 when the reference expands from twenty to fifty chains. However, this subset contains only five participants and cannot support reliable population-level inference.
Appendix L; Appendix M; Appendix Table 19. The variance-head-free checkpoint is intended to demonstrate compatibility with a denoiser trained without a learned variance head. However, Appendix L states that this experiment uses the observed blur difference in the conditioning vector, whereas the principal deployment experiment zeroes the blur-difference component because it depends on information from both scans. Thus, the head-free experiment changes both the training objective and an important aspect of inference conditioning. Although the comparison demonstrates that T-PT can run without a variance head, it does not cleanly isolate the effect of removing that head on uncertainty-ranking performance.
Section 5.3; Appendices J and K. The paper reports that the Jacobian-based estimators exhaust memory on a 40 GB accelerator, while T-PT completes successfully. This is useful practical evidence, but Appendix K also explains that activation checkpointing was disabled to make forward-mode differentiation possible with the released implementations. Therefore, the observed memory failure is not a general impossibility result for Jacobian-based uncertainty estimation.
We thank the three reviewers. Two of them asked for measurements that the submission did not contain, and the third for results that it reported only in its appendices. We took advantage of this discussion period to make the former and to bring the latter into the main text. They are in the revised PDF and are summarised here for the area chair and for all reviewers.
New measurements
Summary of changes in the revised PDF. Changed text is in blue. The numbering of the original sections, tables (1-23), appendices (A-U) and equations is unchanged; all new material is placed last (Apps. V-X, Tables 24-27) or in App. Q (Fig. 4).
Changes of substance
Changes of wording only. The remaining blue text renames headings and rewrites sentences for clarity, in the main text and the appendices. No number, reference or formula was changed by these edits.