- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper develops a metric called "Denoising Strain" which measures how the local geometry changes during denoising using diffusion models. The authors explore how amplifying certain directions during denoising by perturbing the denoising trajectory can significantly change the generated images' semantics, while authors also note how anisotropy develop in diffusion models during training.
Critical strengths
- The paper develops a new metric for assessing how the geometry of diffusion models changes during sampling and training.
- The authors follow a well-known framework (EDM) in their evaluations.
- The authors describe the method and experiments clearly and concisely.
Critical weaknesses
I find the significance/usefulness of Denoising Strain unclear. The paper shows that certain directions in the sampler are more sensitive to perturbations than random directions, and that anisotropy during denoising develops early in training. However, the paper does not adequately establish how early anisotropy relates to meaningful performance gains.
More generally, it's unclear what new thing Denoising Strain tells us that can't already be gleaned from existing diagnostics or simpler Jacobian-based analyses. The paper doesn't demonstrate how the proposed measurements could be used to improve or guide training, predict model quality, or better understand the distribution of generated images. Given the limited theoretical novelty, experimental scope, and lack of meaningful comparisons with related methods, I find the paper doesn't demonstrate significance of contributions.
Major weaknesses
- Unclear significance/practical usefulness of Denoising Strain. The paper introduces Denoising Strain to understand how structure develops during training, but does not clearly establish what useful information it provides beyond existing diagnostics. Specifically, it is unclear whether Denoising Strain can predict model performance, guide training, or identify problems that would otherwise be difficult to detect. (lines 34-47, 456-468)
- No direct evidence that early anisotropy corresponds to meaningful learning. The authors show that perturbing in strain directions can produce meaningful changes to generated images for pretrained models (Sec 3), and separately show that anisotropy emerges early during training (Sec 4). However, they do not demonstrate that the early-appearing directions already capture meaningful semantic or distributional structure. In my view, the claim that early geometric organization and learning are connected is not justified by these experiments. (lines 270-323, 351-387, fig 3,4,8)
- Insufficient comparisons with other ways of measuring learning. The paper doesn't properly investigate whether other measurements (eg denoising loss, Jacobian spectral statistics of the denoiser, or simple measures of generated-image structure) detect signs of learning equally well. Although held-out loss and image morphology are reported for MNIST, there is no systematic comparison of early-detection across metrics. Denoising Strain changing early in training does not necessarily make it a better indicator of future generative performance. (lines 341-349, 370-387, fig 4,9, appendix D.1-D.2)
- The authors did not cover related works comprehensively, particularly on Jacobian-based interpretability of diffusion models. The related works section omits several papers that already use denoiser Jacobians to characterise semantic directions or generative behaviour or perform image editing [1] [2]. (lines 444-452)
- Additionally to 4, authors do not provide meaningful comparisons against exisitng Jacobian-based interpretability methods. This paper's main sampling-time experiments compare Denoising Strain directions with random perturbation directions, instead of methods proposed in earlier works. It is unclear whether computing Denoising Strain through sampler steps provides benefits over analysing the denoiser model's Jacobian directly, especially given conceptual overlap with earlier work. (lines 174-197, 207-226, table 1, appendix B.1)
- Authors provide limited evidence that the directions from Denoising Strain correspond to meaningful semantic structure. Fig 3 has visualisations, but the main quantitative metric (in the main body) is pixel-space separation between generated images, which doesn't measure semantic difference or sample quality. The feature-prediction experiments report only 0.62-1.72% and 0.66-2.17% explained variance in DINO features on AFHQ and FFHQ, respectively. The authors also report significant feature-space responses in fig 7, but these results don't clearly establish that the directions correspond to interpretable semantic changes. (lines 270-323, 790-809, fig 3,6,7, appendix C)
- Limited generality of experiments. Training-time results are limited to MNIST and CIFAR-10, with a few trajectories for measuring anisotropy. Sampling-time experiments use pretrained AFHQ/FFHQ models. All experiments use EDM in pixel space, so it is unclear whether the observations generalize to somewhat more modern SiT/JiT/DiT frameworks ([3] [4] [5]), larger-scale generation (ie higher-resolution images, more trajectories, or text-to-image), or other modalities (eg text to img). The paper does compare Euler/Heun, but only on selected MNIST checkpoints. The authors indicate "Diffusion Transformers provide a direct next setting; language models..." in the future works section around line 464, but this is insufficient since the experimental results are significantly lacking in both depth and breadth for an interpretation paper. (lines 207-226, 333-344, 461-465, appendix B.2)
- The claim that during training, anisotropy provides information beyond FID is not adequately established. The experiments mostly show that geometry and FID change on different timescales, rather than demonstrating what additional information is provided by anisotropy measurements about the generated distribution/model performance. Specifically, it's unclear what changes in the generated image distribution correspond to the emergence/strengthening/other changes in anisotropy. (lines 370-426, fig 4,9, appendix D.4)
- The paper does not adequately establish whether geometric measurements remain useful for evaluating model quality at later stages of training, particularly when the model approaches its best FID. The paper emphasises early training when anisotropy first emerges, rather than whether it remains informative as a model approaches its best FID. The late-training MNIST comparison uses similarity to the final checkpoint's geometry, and this isn't an independent measure of quality, since becoming more similar to the final checkpoint does not necessarily mean becoming a better generative model. There is insufficient analysis of whether Denoising Strain discerns high-performing checkpoints form low-performing ones or whether it explains changes in the models' performance. (lines 370-426, fig 4(d),9, appendix A,D.4, eq 13)
- The authors primarily rely on FID for their training-time results. It can be useful to evaluate whether the conclusions hold with distributional metrics computed in different representation spaces (see [6]). Since authors claim that Denoising Strain reveals aspects of generation that FID and other metrics miss, using different latent spaces for distributional metrics is important. (lines 341-344, fig 9, appendix D.1)
Minor weaknesses
- Small sample size for the sampling-time intervention experiment (Sec 3). Table 1 uses only ten trajectories per dataset, which is insufficient to establish how consistently the effects occur across generated images. The authors conduct a larger experiment with 100 held-out trajectories, but this uses a different formulation and different perturbation method.
- The paper has limited theoretical novelty. The central derivations are largely standard and lack insight, while various properties largely follow from standard facts from linear algebra and dynamical systems. The paper doesn't provide a theoretical relationship between the tensor order parameter and performance/generalisation.
- Computational overhead isn't properly justified via utility/efficiency. Computing the leading strain directions requires repeated jvp/vjp, which is time-consuming based on my experience despite optimised PyTorch-native functions. Appendix B reports ~353-366 seconds per trajectory for 9 denoising states. Without a practical application and a comparison against other diagnostics, it's unclear when this additional computation due to Denoising Strain would be worthwhile.
Questions for authors
See weaknesses.
[1] Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models, https://arxiv.org/pdf/2303.11073
[2] Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing, https://arxiv.org/pdf/2409.02374
[3] SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers, https://arxiv.org/pdf/2401.08740
[4] Back to Basics: Let Denoising Generative Models Denoise, https://arxiv.org/pdf/2511.13720
[5] Scalable Diffusion Models with Transformers, https://arxiv.org/pdf/2212.09748
[6] Representation Fréchet Loss for Visual Generation, https://arxiv.org/pdf/2604.28190