acceptodds
Under review as a conference paper at ICLR 2027

Your Generative Model Is Secretly a Data Dimensionality Estimator

Abstract

Local intrinsic dimension (LID) quantifies the local degrees of freedom of the data: the independent directions in which an object can vary while remaining on the data manifold. Classical estimators infer it from nearest-neighbor statistics and can require prohibitively many samples in high dimensions. Methods such as LIDL and FLIPD instead recover it from how a generative model's density changes with the scale of added Gaussian noise. Their formulas and guarantees, however, are tied to Gaussian perturbations and to a specific model output, whereas many domains train generators with different distributions. It remains open when a model already trained for its own domain yields a justified LID estimate, and at what cost. Our main result extends this density-slope criterion beyond Gaussian noise: it gives sufficient conditions on the perturbation kernel and the local data geometry under which the logarithmic scale derivative of the perturbed density recovers LID as the noise vanishes. This derivative equals the response of the posterior-mean reconstruction of the clean point to small shifts of the noisy observation, plus a correction. The correction is required at a boundary of the support but vanishes at interior points of a smooth data manifold, where the response alone recovers LID for Gaussian and admissible non-Gaussian kernels alike. Whenever a model's output determines this reconstruction, the response is computed with a few Jacobian–vector products and no additional training. Specializing the criterion gives explicit LID estimators for diffusion, independent affine flow matching including non-Gaussian sources, generalized Brownian Schrödinger bridges and scale-conditioned normalizing flows, recovering FLIPD and Gaussian flow-matching estimators as their Gaussian special cases. On LID benchmarks with Gaussian and non-Gaussian perturbations, the response alone is competitive with the full estimate and more robust when the learned correction is inaccurate; on pretrained crystal and protein models, including ones trained with non-Gaussian noise, it tracks lattice symmetry and side-chain flexibility.

Primary area:
generative models
Keywords:
Local Intrinsic Dimension Estimation, Density Estimator, Normalizing Flows, Flow matching, Manifold learning, Diffusion models, Local intrinsic dimension

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HpNhOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the question under which conditions a trained generative model can be used to estimate the intrinsic dimensionality dd of the data manifold. Perturbing this data-manifold with Gaussian (or uniform) noise with variance ε\varepsilon results in noisy observations with density pεp_{ \varepsilon}. Existing methods estimate dd by calculating ∂εlog⁡pε\partial_{\varepsilon} \log p_{ \varepsilon} which equals to d−Dd-D under suitable conditions. The paper extend this density-slope method to non-Gaussian noise kernels (Theorem 2.2). Moreover, they derive conditions under which the Jacobian of the posterior mean can be used for estimating dd (Corollary 2.4).

The authors derive explicit expressions of the density-slope estimator for different generative models (Table 1) and compare their performance on different toy benchmarks. A given generative model with non-Gaussian perturbation kernel can be used to estimate dd without retraining as demonstrated on two real-world examples from chemistry and biology.

Critical strengths

State-of-the-art generative models such as diffusion models add noise to the data manifold in a controllable manner; the noise kernel is typically Gaussian but can be non-Gaussian in some applications. In the latter case, using the density-slope method for estimating dd requires retraining models with different noise magnitudes. The master Theorem, Theorem 2.2, generalizes the density-slope method to non-Gaussian kernels allowing to estimate dd without retraining. For practitioners, Table 1 provides a useful look-up table to turn a trained generative model to an intrinsic dimensionality estimator. The decomposition of the density-slope estimator as response term plus correction is interesting and potentially useful to study differences of generative models.

Critical weaknesses

The paper lacks clarity on different levels. Mathematically, for the theory crucial assumptions are either only described or deferred to the exhaustive appendix (A1, A2, and A3). How restrictive those assumption are - and which models/kernels fulfill those assumption or not - is not discussed in the main text. Intuitions for the proofs are not provided such that without reading the appendix it is difficult to assess correctness of mathematical results.

Section 4.3 starts with the question whether a trained model for its own domain provide useful local geometry through response. But useful information can be deduced from the density-slope method alone with diffusion models providing the necessary score without retraining as well. In Table 2, for the Gaussian models, the density-slope method (referred to as full) performs better than the response method except for the rectified flow. A principled discussion when to use response or when to use the density-slope method is missing (see Q1 below). Figure 1 is a good starting point for providing intuition, but in practice, whether a point is interior or a boundary point is unclear.

The experimental results in Section 4 are difficult to comprehend. LID-Benchmarks are not explained. Justifications for choosing the specific model families (Table 1) are not provided. MAE score in Table 3 is not clearly explained. Results are of qualitative nature; knowing that lower or higher is better helps comparing the different models, but to which extend the task of estimating dd is solved is left unanswered. The only true reference is a constant estimator d^=1\hat{d}=1. Overall, Section 4 is very dense in information. For example, authors write "For known LID, a disjoint source-train holdout selects each checkpoint’s scale by primary-readout MAE." which I could only understand with the appendix. But this sentence is crucial as it basically says that the known dd is used for estimating dd itself. Other expression such as "primary readout”, “dependent readouts”, “inherit Full’s selected scales”, “suite-balanced relative-MAE aggregates”, “pointwise Kneedle” require plain language descriptions to be understandable. So does Section 4.3 which is inaccessible for non-expert on the crystals or proteins.

Questions for authors
  1. Assume practitioners have a problem which can be solved using a generative model. They are interested in local informations as well. Which generative model should they use and why? Which ID method, density-slope or response, should they use and why? What are the determining factors of the underlying problem determining the choice?

  2. In Section 4.3, why did you just report the response and not the density-slope? DiffCSP is a diffusion model so score should be available? Or do I misunderstand something?

  3. Which noise kernels fulfill the necessary conditions for the theory to apply? You mentioned a few in the introduction (e.g. wrapped-normal), are those admissible? How restrictive are the assumptions more generally?

  4. Please explain in very simple terms what experiments 4.3 is measuring. What is the data dimension DD, what is the ID dd. What are the noise kernels. Do they fulfill the necessary assumptions yes or no. What are the estimates d^\hat{d}. How do they compare with other methods.

Further clarification requests:

  • Line 138-139. It is not clear to me why a projection onto dd yields a Jacboian trace of dd. Please explain

Suggestions/Remark:

  • Line 124 is confusing as stated and I think even wrong. The local dimension does not describe how the probability of finding a data points changes as radius tends to zero; this depends on the probability itself (uniform with flat curvature will not change).
  • I think the paper would benefit from providing more intuition. After Proposition 2.1, Figure 1 could explain the involved terms and their differences. Then, briefly introducing two important generative models - diffusions and flow matching - and explaining what they provide (score or velocity). This motivates why both estimators are important to be unbiased (Proposition 2.1 itself does not).
  • I'm not sure how useful a Table 1 is for the main text. I'd rather see one clear derivation sketch for a standard model (e.g. diffusion) together with a discussion on the conditions under which it converges, computational complexity etc.
  • I suggest motivating the different model variants. Why did you decide using those and not others?
  • I'm not sure if the title accurately addresses the papers contribution. The density-slope method already suggests that generative models are data dimensionality estimators. .
Reviewer_NRXZOpenReview ↗
Rating
1
Confidence
1(−1)
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper, as far as I can understand, aims to estimate the local dimensionality of the data manifold under the assumption that high-dimensional data (like images in pixel space) lives on a subspace of lower dimensionality. One possible strategy to estimate such local dimensionality is through a trained generative model, by perturbing it with noise and evaluating how the learned density changes. The proposed gap here is that, instead of a Gaussian noise perturbation, to generalize to non-Gaussian perturbations for trained generative models in domains that have physical constraints for the type of noise realizable, such as in molecular dynamics. The rest of the paper appears to prove a similar relationship between noise scale and the change in density (density-slope criterion) for a number of non-Gaussian noise settings or generative modeling frameworks.

Critical strengths
  • the paper addresses a fundamental problem in density estimation and generative modeling, i.e., getting an accurate estimate of data manifold dimensionality
  • the paper appears to be comprehensive, proposing a “master theorem” and local dimensionality readout for a variety of generative model families, including previously derived relationships such as for diffusion models.
Critical weaknesses
  • to be completely honest, I’m not sure what the authors did, or how it was done, in this paper.
  • the text is almost entirely incomprehensible to a human. The first half of the introduction is surprisingly coherent, giving a broad and somewhat digestible overview of local dimensionality estimation methods, including nearest-neighbor based ones and learned generative models based methods. But from line 80 onwards, the ratio of jargon to plain English prose increases exponentially. The same is reflected in the abstract, where many terms are simply dropped in there without reference or care.
  • in other words, the writing impedes understanding to such an extent that implies either that 1) I have basically no background required to evaluate this paper, or 2) it was written with such little care that at any other time in history, this would be an unaccepted scientific paper. My scores reflect the latter, though the former is always possible.
Questions for authors
  • Could the authors unpack the abstract in their own words? What is the density-slope criterion, perturbation kernel, and the response of the posterior-mean reconstruction of the clean point? Why was it deemed unnecessary to define these terms before they are used?
Reviewer_hyHXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies local intrinsic dimension estimation using outputs of generative models. It extends the density-slope criterion to admissible non-Gaussian perturbations and relates this criterion to the Jacobian trace of the posterior mean, called Response, plus a correction term. At regular interior points of a smooth manifold, the correction vanishes as the noise decreases, so Response alone recovers the local dimension. This allows estimation from certain non-Gaussian flow-matching velocities without an additional score model. The paper evaluates these readouts on known-dimension benchmarks and presents preliminary applications to pretrained crystal and protein models.

Critical strengths
  1. The non-Gaussian extension has a concrete methodological benefit. Recovering dimension from Gaussian density scaling and denoiser derivatives is already established. The useful contribution here is the justification of Response for admissible non-Gaussian kernels. For general non-Gaussian flow matching, the velocity determines the posterior mean but does not directly provide the score needed by Full. Corollary 2.4 therefore allows dimension estimation from an available model output without fitting an additional score model. I consider this a meaningful extension of the existing results. The logistic-source experiment provides direct evidence that this simpler readout can retain useful accuracy.

  2. The relation between Response and Full gives a useful interpretation of the estimators. Proposition 2.1 and Corollaries 2.3–2.4 explain how reconstruction sensitivity is related to local dimension and when the correction can be omitted. The distinction between smooth interior points and boundaries helps clarify the scope of the simpler estimator. Table 1 also makes the connection to different model outputs explicit. These results provide a coherent account of several estimators and identify conditions under which they have the same limit.

Critical weaknesses
  1. Practical use still depends on scale calibration. The stronger results on the known-LID benchmarks use scales selected with labeled holdout data. The tested label-free selector performs substantially worse; on IDR and Arrows, every Kneedle result is worse than the constant estimate. The authors explicitly acknowledge this limitation. It does not undermine the asymptotic results, but it limits the practical conclusions for settings where suitable calibration data are unavailable.

  2. Evidence on the relative reliability of Response and Full remains limited. The paper reports that their relative accuracy varies across tasks and provides a useful diagnosis of the Student failure cases. However, the benchmark comparisons use one training seed and do not quantify variation across retraining. The results support the usefulness of Response under the tested protocol, while providing more limited guidance on when to prefer it in a new setting.

Questions for authors
  1. Could the authors briefly comment on the scale sensitivity of logistic Response, including whether useful accuracy holds over a broad range?

  2. Could the authors briefly clarify whether the discussion in Section 4.2 offers guidance on choosing Response over Full when the true dimension is unknown?

  3. Could the authors briefly clarify in the main text how lattice Response should be interpreted in relation to the symmetry-derived degrees of freedom in Table 3?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.