acceptodds
Under review as a conference paper at ICLR 2027

Understanding Latent Diffusability via Fisher Geometry

Abstract

Diffusion models often degrade in latent spaces, yet the formal causes remain poorly understood. We quantify latent-space diffusability via the rate of change of the Minimum Mean Squared Error (MMSE) along the diffusion trajectory. Our framework decomposes this MMSE rate into contributions from Fisher Information (FI) and Fisher Information Rate (FIR). We show that isometric embeddings preserve intrinsic FI and establish quantitative intrinsic-FI bounds for a broader class of bi-Lipschitz encoders with controlled weak volume distortion, whereas FIR is governed by the interplay between encoder and data geometries. Our analysis separates four geometric contributions in local stability bounds for Gaussian-smoothed FIR: dimensional compression, tangential distortion, high-frequency encoder curvature, and curvature of data manifold. Experiments across diverse autoencoding architectures provide qualitative support for the geometric mechanisms identified by the theory and show that empirical FI and FIR track several measures of generation quality and latent-space geometry in the settings tested. We establish FI and FIR as a comprehensive analytical framework for understanding latent diffusability.

Primary area:
generative models
Keywords:
diffusion models, latent diffusion, autoencoders, latent geometry, Fisher information

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QwHxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies latent diffusability through Fisher information (FI) and its dissipation rate (FIR) under Gaussian smoothing. It relates these quantities to the derivative of the population MMSE, establishes intrinsic-FI bounds under geometric encoders, and develops local FIR stability bounds involving dimensional compression, tangential distortion, encoder nonlinearity, and manifold curvature. Experiments on Gaussian toy distributions and image representations use trained diffusion models to estimate FI/FIR and examine their relationship with representation geometry and generation quality.

Critical strengths

The paper tries to address a theoretical question: how do changes in encoder geometry affect FI and FIR? In particular, the intrinsic-FI transformation bounds distinguish tangential metric distortion from variation in the volume Jacobian, while the local Gaussian-smoothed FIR bounds account for near-isometry, encoder nonlinearity, and manifold curvature. These results provide a more quantitative account of representation geometry than an informal appeal to “latent distortion.” I regard this geometric analysis as the paper’s principal strength and as potentially valuable independently of its practical estimation procedure.

Critical weaknesses

1. The empirical FIR estimates disagree substantially with the exact Gaussian benchmark.

In the dimension-varying experiment in Section 4.2, two standard Gaussian coordinates are embedded in D=512D=512 dimensions, and the encoder preserves these coordinates while removing only zero coordinates. This setting changes ambient dimensionality without losing signal or introducing nonlinear distortion.

For this setting, Eqs. (39)–(42) give

Rk(τ)=2(1+τ)2+k−2τ2,R_k(\tau)=\frac{2}{(1+\tau)^2}+\frac{k-2}{\tau^2},

and

DRsc(τ)=∣1−d−2512−2R512(τ)Rd(τ)∣.D_R^{sc}(\tau)=\left|1-\frac{d-2}{512-2}\frac{R_{512}(\tau)}{R_d(\tau)}\right|.

At d=100d=100 and τ=0.01\tau=0.01, the exact normalized deviation is approximately 1.62×10−61.62\times10^{-6}, whereas the corresponding blue curve in Fig. 2(b) appears to be approximately 0.80.8. Both the panel’s ordinate and the accompanying text identify this quantity as DRscD_R^{sc}.

Appendix C explains that these quantities are estimated using learned-score Jacobians. The discrepancy therefore raises a central calibration question: how accurately do these estimates recover the population quantities analyzed by the theory? Agreement with the decreasing trend of the loose upper bound (D−d)/(D−m)(D-d)/(D-m) does not resolve the disagreement with the exact value.

Until this discrepancy is explained, the empirical curves provide insufficient support for the proposed geometric interpretation. Their variation may reflect score-Jacobian estimation error or normalization differences as well as representation geometry.

2. The incremental diagnostic value of FI/FIR remains unclear.

The experiments compare FI/FIR curves and generation performance after diffusion models have already been trained. They do not establish whether these diagnostics can reliably inform encoder selection or design. Reliable pre-training estimation is explicitly left to future work. The current experiments therefore demonstrate retrospective associations, but do not establish that FI/FIR can guide representation selection before expensive diffusion training.

The comparisons also do not establish what FI/FIR reveals beyond latent dimensionality, scale, reconstruction quality, or spectral statistics. Although the paper uses similarly sized pretrained models and normalizes latents in the FFHQ experiments, these measures do not isolate the additional explanatory value of FI/FIR.

The geometric bounds are potentially informative, but the experiments have yet to establish a concrete decision or insight that these diagnostics enable beyond the simpler measurements already available. This limits the demonstrated practical significance of the framework.

Questions for authors
  1. Calibration of Fig. 2(b): For D=512D=512, d=100d=100, m=2m=2, and τ=0.01\tau=0.01, Eq. (42) gives DRsc≈1.62×10−6D_R^{sc}\approx1.62\times10^{-6}, while Fig. 2(b) shows approximately 0.80.8. What accounts for this difference? Please report the underlying estimates of R512R_{512} and RdR_d, specify the normalization and noise parameterization used, and explain whether the same issue affects the other FI/FIR comparisons.

  2. Incremental diagnostic value: What do FI/FIR reveal about representation quality that is not already explained by dimensionality, scale, reconstruction quality, or spectral statistics? Which existing results isolate this additional information, and what conclusions about encoder selection or design do those results support?

Reviewer_JFmNOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors study diffusion models in latent spaces. Specifically, they propose to use information-geometric tools to analyze which encoders produce latent spaces better suited for diffusion models, or "easier to diffuse". Empirical analyses suggest that the proposed geometric estimators indeed correlate with diffusion's ability to accurately model the given distribution.

Critical strengths

I think that the main strength of the paper is that the question being studied is genuinely interesting, and I think it is important. Diffusion models are indeed usually applied in latent spaces so it would be nice to understand how to best design these encoders to optimize performance.

Critical weaknesses

I see two main weaknesses of the paper

  1. Presentation - the paper reads as a dry technical report. Technical results are being presented without much intuition being conveyed or a story being told. It was difficult for me to follow, and I struggled to see why certain results are relevant and what the takeaways are.
  2. Empirical validation - the authors evaluate the proposed geometric quantities on some toy examples and a few image examples. The results indeed seem to suggest that FI and FIR correlate the model's quality metrics. However, I have two concerns there. First, the authors claim that the encoder is being evaluated, and yet the estimator requires the score of the diffusion model. I see this as a confounding factor and perhaps a different diffusion model trained on the same representation would give very different estimates of FI and FIR. Second, I do not feel like this analysis is actionable. I do not see a recipe that will tell me how I should construct the auto-encoder which will improve my diffusion model
Questions for authors
  1. Can the authors comment on how reliable are the FI/FIR estimates since they require the score function trained on a given representation? In theory, they are only accurate if the diffusion model is perfect. And a perfect model obviously models any distribution well
  2. what are "activation interfaces"?
  3. Can the authors comment on relevance of Theorem 3.3?
  4. Lines 278-280 - can the authors comment on the relevance of this bound? Or the one in lines 288-289?
  5. I was excited by "Geometric Interpretation of FIR Deviation" to gain more intuitions about the presented results, but I do not feel like it helps interpret the results. The authors comment on four terms, but use language that is hard for me to understand or leaves the interpretation to the reader. For example, one of the terms is "Local metric stretching or compression", but it is unclear to me why this is relevant.
  6. What is "low-noise inflation" ? And what do the authors mean by "global folding, or interactions between multiple local branches." ?
  7. What does "The exact normal-direction contribution of dimensional compression on flat supports and the leading local compression contribution on curved supports." mean?
  8. The authors discuss "Curvature-Dependent Penalty", but conclude with "The resulting term combines intrinsic manifold curvature with the encoder’s sensitivity to normal displacement.". What does it mean? What is the interpretation, and how is it actionable?
Reviewer_1zFgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

As I understand it, the paper wants to study how "diffusable" are latent embeddings. Instead of training a diffusion model on some data space (for example, pixels), a diffusion model is being trained on the latent embeddings, obtained through an autoencoder. The authors are using the MMSE and its derivative with respect to the noise vraiance (MMSE'). They decompose this MMSE' into three terms, including the Fisher Information (FI) and its derivative (FIR). They authors give bounds and conditions under which an encoder approximately preserves those FI and FIR metrics.

Critical strengths

The theorems and proposition appears correct, the limitations are well stated. The authors are open about several limitations.

Critical weaknesses

My main concerns are:

  1. The paper does not show that diffusability relates to "learning" difficulty or generation quality. The authors describe diffusability as how quickly the denoising error increases as more noise is added (MMSE'). My main concern with this definition is that it is unclear how useful it is in practice, as it is not related to sample quality, efficiency of the generative model, or even faithfulness of the "latent" generative model with respect to the "data" one. From this definition, that is defined mathematically but carry too little physical intuition, FI and FIR are used as metric capturing some performance of the generative models. Additionally, Figure 11 appears to give the VAE latent the lowest MMSE slope at small noise, despite its having the worst generation quality.

  2. The quantities are misinterpreted or underexplained. First, I believe the “Fisher geometry” framing is misleading, since the FI used is the Fisher information of a density (which is a derivative in x), but not Amari’s Fisher–Rao metric on parameters (despite Amari (2016) and Karczewski (2016) being cited while introducing their scope). The theorems concern local, intrinsic, population quantities but are only checked qualitatively against global, ambient, learned ones. This is a problem, because the authors are claiming that an encoder failing to preserve geometry (curvature) locally is the cause of a bad latent-space quality. But FIR is only one term of the slope of the ideal MMSE. An increase in FIR increases MMSE' (also called difficulty) only if FI and the dimension do not change. But an encoder changes R, FI and the dimension at the same time. Thus, a change of curvature, and so FIR alone, does not show that the latent space is more difficult to denoise, as the paper defines it. For example, zero padding changes FIR, but the MMSE' does not change.

  3. The paper would benefits from some comparison with the litterature In particular, "Information-Theoretic Diffusion" by Kong et al (2023) is one prior paper to use the notion of MMSE linked with noise to study diffusion models from an information theoretic view.

Questions for authors

Why should a the "latent" FI and FIR match the "data" FI and FIR? How are those quantities linked to sample quality? In Fig. 11, the VAE latent has the lowest MMSE slope, so it is the most diffusable by your definition, yet it has the worst FID (62.51). Can you re-explain intuitively what diffusable mean, and why we should pay attention to it?

The theorems bound local, intrinsic, population quantities, but the experiments measure global, ambient, learned ones. Can you state a global result, or test the local bounds directly?

Can you show, or explain, your theory and claims with a geometric dataset much easier to visualise and understand, like a sphere in a 3 dimensional space, and its latent space could be visualised in 2d? With different autoendoder, FI, FIR... we could see and, so better undertsand, the impact of FI, FIR, on both the latent embeddings and the sphere.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Changes in the Revised Manuscript

We thank all reviewers for their constructive feedback. We have uploaded a substantially revised manuscript, with changes marked in blue. The main revisions are summarized below.

  1. Improved theoretical motivation and presentation (Sections 2–3). We clarified the motivation for studying latent diffusability through FI and FIR, expanded the explanation of pixel–latent comparisons, and reorganized the theoretical results to highlight their main conclusions and geometric interpretations. Precise assumptions and proofs remain in the appendix.
  2. Clarified the scope of FI, FIR, and MMSE (Sections 2–3 and Appendix B). We distinguished population denoising sensitivity from neural score-learning difficulty and generation quality, clarified that FI and FIR must be interpreted jointly in the MMSE decomposition, and distinguished spatial Fisher information from the Fisher–Rao metric.
  3. Expanded theoretical and numerical validation (Section 4 and Appendices E–F). We added exact Gaussian comparisons, revised the dimension-varying experiment, and analyzed the original high-dimensional setting, including its remaining estimation limitations (Appendix F.1). We also added a direct spherical-cap evaluation of local FIR stability using Gaussian-mixture calculations (Appendix F.2, Figures 9–12).
  4. Additional analyses and related work. We included dimension- and SNR-normalized MMSE comparisons and score-estimator robustness analyses (Appendix F.3.2, Figures 17–18), and expanded the discussion of related information-theoretic work, including Kong et al.

We have also refined our claims to better reflect the scope of the theoretical guarantees and empirical evidence. We hope these revisions make the contributions and limitations of the work clearer and easier to assess.

AuthorsOpenReview ↗

Manuscript Further Revised

Thank you again for your constructive feedback. We have now updated the revised manuscript, with changes marked in blue.

We expanded the motivation for using FI and FIR to study latent diffusability and clarified the rationale for comparing pixel- and latent-space quantities. We also substantially improved the presentation of the theoretical results in Section 3, including their assumptions and geometric interpretations. In addition, we clarified the relationship between MMSE, FIR, and diffusability, distinguished spatial Fisher information from the Fisher–Rao metric, and expanded the discussion of related work, including Kong et al.

We hope these revisions make the motivation and theoretical contributions clearer.

AuthorsOpenReview ↗

Manuscript Further Revised

Thank you for emphasizing the need for substantial improvements to the presentation. We have continued working on these issues since our initial response and have now uploaded a substantially revised manuscript, with changes marked in blue.

Sections 3.1–3.3 have been substantially revised to clarify the motivation, assumptions, and interpretation of the theoretical results. We also distinguish population denoising sensitivity from the difficulty of learning a neural score model and from generation quality.

We have incorporated additional analyses, including the original-setting calibration study and its remaining limitations (Appendix F.1), a direct spherical-cap comparison using Gaussian-mixture calculations (Appendix F.2, Figures 9–12), and dimension/SNR-normalized MMSE comparisons and score-estimator robustness tests (Appendix F.3.2, Figures 17–18).

These revisions incorporate the main explanations and qualifications into the manuscript itself. We hope the revised presentation and specific references make the contributions and limitations easier to assess.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.