acceptodds
Under review as a conference paper at ICLR 2027

Latent Manifold Dimension from Noisy Observations: Identifiability and Recovery via Diffusion Scores

Abstract

Diffusion models trained by denoising score matching (DSM) offer a promising approach to intrinsic dimension (ID) estimation. However, when observations are not exactly supported on a manifold, two questions remain: **(Q1)** Does the observed law determine a unique latent dimension, and **(Q2)** which diffusion scales recover it? We address these questions under bounded normal tubular noise, allowing a broad range of noise distributions. For (Q1), we define *latent manifold dimension* (LMD) relative to a prescribed model class and establish a uniform reach-separation condition for identifiability that cannot generally be relaxed at its boundary. To study recovery, we first introduce *local Hessian spectral dimension* (LHSD) as a score-based dimension statistic and examine how accurately it can be estimated from data at a given diffusion scale. Under suitable regularity and thin-tube conditions, we establish its minimax estimation rate, attained by an unrestricted empirical-DSM reference estimator. We then address (Q2): LHSD can fail at fine scales and tends to zero as the diffusion scale grows, while a sharp approximation bound characterizes recovery at intermediate scales. Experiments with diffusion transformers qualitatively support this scale dependence, including accurate estimation of the latent dimension at intermediate scales.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Intrinsic dimension, Latent manifold, Diffusion score, Hessian spectral analysis, Manifold learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DeP3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

This paper theoretically studies when the latent manifold dimension can be uniquely defined and recovered from noisy observations. It first establishes an identifiability result showing that, under bounded normal noise, the latent dimension is uniquely determined when 2ξ0<τ02\xi_0<\tau_0, and also provides counterexamples at the boundary. It then analyzes the existingLID estimator, LHSD, across diffusion scales, showing that very small scales can capture noise-induced directions, while very large scales erase dimension information; at suitable intermediate scales, an affine-calibrated version of LHSD recovers the latent dimension with a sharp bias tradeoff and an optimal scale of approximately σ2≍ξ0τ0\sigma^2\asymp\xi_0\tau_0. The paper further derives matching finite-sample minimax rates for estimating the finite-scale LHSD field. These theoretical results are validated on synthetic tubular manifolds using DSM-trained DiTs, where the experiments reproduce the predicted fine-, intermediate-, and coarse-scale behavior and show accurate dimension recovery at appropriate intermediate scales.

Critical strengths

S1. Timely theoretical explanation of an important empirical phenomenon in diffusion-based dimension estimation.
Recent generative-model-based LID estimators, such as LHSD and FLIPD, have shown the potential to substantially improve over traditional kNN-based estimators and have attracted increasing attention. Previous empirical studies have suggested that their estimation accuracy can vary dramatically with the diffusion noise scale, but the mechanism behind this scale dependence has remained unclear. A major strength of this paper is that it provides a theoretical explanation for this phenomenon, clarifying why different diffusion scales can lead to qualitatively different dimension estimates.

S2. Strong and coherent theoretical framework for latent-dimension identifiability and recovery.
The paper first asks the fundamental question of whether the latent manifold dimension is identifiable from noisy observations, establishing the clean condition 2ξ0<τ02\xi_0<\tau_0 together with a counterexample at the boundary. It then characterizes how the existing LHSD estimator behaves across diffusion scales: fine scales may capture noise-induced directions, intermediate scales recover the latent dimension, and excessive smoothing removes dimension information. The resulting sharp bias tradeoff and the population-optimal scale σ2≍ξ0τ0\sigma^2\asymp\xi_0\tau_0 provide a particularly intuitive picture of when dimension recovery is possible.

S3. Substantial statistical guarantees complement the geometric analysis.
Beyond the population-level recovery results, the paper derives matching minimax rates for estimating the finite-scale LHSD field. By explicitly separating structural identifiability, approximation bias, and finite-sample estimation error, the analysis provides a fairly complete theoretical account of the different sources of difficulty in recovering latent dimension under the tubular-noise model.

Critical weaknesses

W1. From a practitioner’s perspective, the importance of diffusion-scale selection is not itself a new finding.
Prior work on FLIPD and LHSD had already documented strong sensitivity to the diffusion noise scale and proposed practical heuristics or diagnostics for selecting suitable scales. Therefore, the practical message that inappropriate scales should be avoided was already known. The main novelty of this paper is instead the theoretical explanation of this phenomenon, through the characterization of fine-, intermediate-, and coarse-scale regimes and the associated bias tradeoff. I think this distinction should be stated more clearly.

W2. There remains a substantial gap between the theoretical estimator and the learned diffusion estimator used in the experiments.
The minimax guarantees are established for an unrestricted empirical-DSM reference estimator, rather than for the DSM-trained DiT used in the experiments. This distinction is particularly important for LHSD because it relies on the score Jacobian/Hessian, and accurate score estimation does not automatically imply accurate derivative estimation. Moreover, the empirical validation is restricted to synthetic tubular-manifold models, and the curved-model references are local-tube approximations rather than exact population LHSD values. Thus, it remains unclear to what extent the theoretical guarantees carry over to practical learned diffusion models and real-world data.

W3. The attribution and novelty boundary surrounding LHSD are insufficiently clear.
LHSD is an existing method introduced in prior work [42], and the paper itself states in the introduction that it “adopt[s] local Hessian spectral dimension (LHSD) [42].” However, the abstract and Section 3 use language suggesting that LHSD is being introduced in the present paper, and the affine-calibrated variant is still referred to simply as “LHSD” in the experiments rather than being clearly distinguished from the original method. The actual methodological novelty appears to be the affine calibration together with the new identifiability, scale-dependent recovery, and minimax analyses. Among the concerns above, I view this as the most serious one, because it directly affects the attribution and novelty of the central method rather than only its practical applicability or empirical validation. The distinction between the existing LHSD method and the contributions of this paper should therefore be made explicit throughout the manuscript.

[42] Genki Osada. Local hessian spectral filtering for robust intrinsic dimension estimation. InForty-third International Conference on Machine Learning, 2026.

Questions for authors

Q1. Can the authors connect the theoretical oracle scale σ2≍ξ0τ0\sigma^2 \asymp \xi_0\tau_0 to an observable, data-driven scale-selection criterion?
In particular, could the existing scale-selection diagnostics used in LHSD, or related heuristics in prior diffusion-based LID methods, be theoretically justified under the tubular-noise model, or at least empirically shown to select scales close to the theoretically favorable intermediate regime? Such a result would substantially strengthen the practical relevance of the theory.

Q2. Could the authors clarify precisely which components of LHSD are inherited from [42] and which are new in this paper?
In particular, is the only modification to the existing LHSD estimator the affine calibration, with the main new contributions being the identifiability and scale-dependent recovery theory? If so, I believe the terminology and novelty claims should be revised consistently throughout the abstract, method section, and experiments.

[42] Genki Osada. Local hessian spectral filtering for robust intrinsic dimension estimation. InForty-third International Conference on Machine Learning, 2026.

11 comments
AuthorsOpenReview ↗

On practical perspective and theory for learned diffusion models

Thank you for pointing out where our wording obscured the contribution. In particular, the submitted abstract used introduce for a statistic developed by Osada [43]. We corrected that attribution throughout the paper. We also revised Section 3 to explain what our theory adds to existing scale diagnostics and Section 4 to state precisely when the empirical-DSM guarantee extends to a learned predictor. Section 5 now evaluates the dimension error at a scale selected from data.

W1

We agree with your assessment. Practitioners already had guidance on scale selection. Kamkari et al. [30] used a knee in the FLIPD curve, and Osada [43] examined whether the LHSD cutoff lies in a gap between Hessian eigenvalue clusters. The revised Introduction credits both approaches and makes our theoretical contribution explicit. Section 3.4 gives matching upper and lower bounds on the worst-case error with which population LHSD approximates the latent dimension at intermediate scales. Under its assumptions, these bounds identify the scale that balances tubular noise and curvature and minimizes this error up to constant factors. This explains why the choice of scale matters even when the score is known exactly.

An important update is that we also added a data-driven diffusion scale selection rule. Section 4.4 gives conditions on the width and location of the interval and on the grid resolution under which the rule selects a scale of order ξ0τ0\sqrt{\xi_0\tau_0} and attains the optimal worst-case population bias order among intermediate scales. In Table 1, adaptive LHSD had lower dimension RMSE than adaptive FLIPD in 15 of 16 settings across the linear, sphere, squiggly, and Swiss roll models under their respective selection protocols. This gives practitioners a way to choose a scale without knowing the tube width or reach, along with an explanation of when that choice is justified.

W2

We agree that the empirical-DSM theorem alone does not establish a rate for the DiT used in our experiments. A model can predict noise accurately at corrupted inputs and still have inaccurate derivatives at the observed queries where we compute LHSD. Section 4.2 now states what would suffice. At a fresh observation, we compare the fitted predictor's symmetrized input Jacobian with that of either the population predictor or unrestricted empirical DSM fitted to the same sample. If the expected squared Frobenius error relative to either reference is O(n−1σ2−d)O(n^{-1}\sigma^{2-d}) uniformly over the model class, the learned LHSD estimator attains the minimax integrated squared-error rate O(n−1σ4−d)O(n^{-1}\sigma^{4-d}) under the conditions of Section 4.1. This condition applies to any differentiable noise predictor, including a DiT or a U-Net.

Figure 10 in Appendix H.7 reports the Frobenius RMSE of the full denoiser Jacobian ID−σ∇wgI_D-\sigma\nabla_w g, normalized by D\sqrt{D}, on held-out queries across scales at n=200,000n=200{,}000. It uses a population reference for the linear model and an empirical-DSM reference for the curved models. We plan to repeat the comparison across training sizes to see how the discrepancy changes with nn. The bound explains why small Jacobian error can lead to accurate dimension estimates at scales where population LHSD closely approximates the latent dimension.

Your concern about the curved-model references matters when interpreting the experiments. For the linear model, we use a numerical population LHSD reference. For the sphere and its radial deformation, the local-tube reference is itself an approximation, so its discrepancy from learned LHSD cannot be attributed entirely to the DiT. The known latent dimensions in these synthetic benchmarks let us measure dimension error directly in Table 1. On the MNIST dataset, the latent dimensions are unknown. Our planned experiments will therefore report how stable the estimates are across scales and how they vary across image classes.

AuthorsOpenReview ↗

Attribution, data-driven selection rule, and affine calibration

W3

We agree that the submitted abstract obscured that attribution. The revised abstract credits Osada at the first mention of LHSD, and the Introduction describes the original estimator and its scale diagnostic. Section 3.2 presents the original spectral filter before defining our affine calibration, the only change we make to the count at a fixed scale. We specify there that subsequent uses of LHSD refer to the calibrated count, and Section 5 identifies the estimator used in the experiments as calibrated LHSD.

We also revised the statement of our theoretical contributions. After establishing when the observed law identifies the latent dimension in Section 2, we treat the two errors in recovery in order. Section 3 bounds the approximation error of population LHSD as a function of diffusion scale. Section 4 then gives the minimax rate for the estimation error when that population field is estimated from finite samples at a prescribed scale. The revision adds a sufficient condition under which a plug-in estimator based on any differentiable noise predictor attains this rate, expressed in terms of the accuracy of its input Jacobian (Section 4.2). We also added a data-driven rule for choosing the scale and showed when it attains the optimal worst-case population bias order among intermediate scales (Section 4.4).

Q1

Kamkari et al. [30] used Kneedle (Satopaa et al. [48]) to find bends in FLIPD curves. In Section 4.4, we apply it to the median LHSD curve computed on independent selection queries. We find the widest stable interval after any detected bend and choose the grid point closest to its midpoint on the log scale. This choice requires no knowledge of the latent dimension dd, the tube width ξ0\xi_0, or the reach bound τ0\tau_0.

Proposition 4.2 explains when this choice works. It requires a sufficiently wide stable interval, with all competing intervals of at least that width lying in the specified scale window and a grid fine enough to resolve their log midpoints. Under the stated model conditions, for sufficiently small ξ0/τ0\xi_0/\tau_0, the selected scale is of order ξ0τ0\sqrt{\xi_0\tau_0} and attains the optimal worst-case population bias rate among intermediate scales. The two-shell example in Appendix G shows that even with exact population counts and the balancing scale on the grid, a stable interval alone cannot guarantee near-optimal bias. Whether the spectral-gap diagnostic of Osada [43] admits a similar guarantee remains open.

On held-out queries, adaptive LHSD achieved an RMSE below one in 14 of the 16 settings across the linear, sphere, squiggly, and Swiss roll models (Table 1). On the d=50d=50 linear model, however, RMSE was 1.9241.924 at the selected scale and 0.4490.449 at the best of the five reported fixed scales. In Figure 4, we will report the gap between RMSE at the selected scale and the lowest RMSE on the full evaluation grid, both computed from the original, unscaled estimates. We use the known dimension only to evaluate the choice.

Q2

Yes. At a fixed scale, we use the same Hessian eigenvalues and fourth-order filter ϕϖ\phi_\varpi as Osada [43]. We still sum the filter weights, but rescale each weight before taking the sum. For an ideal normal projector, the original filter gives each tangent direction weight one and each normal direction weight r=ϕϖ(1)r=\phi_\varpi(1). The rescaling w↦(w−r)/(1−r)w\mapsto(w-r)/(1-r) sends these weights to one and zero, respectively, so the calibrated count equals dd. For the same Jacobian, the two counts differ by at most Dϖ4D\varpi^4, which is 0.00160.0016 for D=256D=256 and ϖ=0.05\varpi=0.05.

The data-driven rule chooses the scale at which to evaluate this count. Our theory addresses when the observed law identifies the latent dimension, how closely the population count approximates it, and how accurately the count can be estimated from a finite sample.

Regarding the response to Q1:

The original LHSD paper already proposed a transition-mass diagnostic for scale selection, but the authors do not compare their Kneedle-based approach against it. Given the relatively large error in the (d=50) setting (RMSE 1.924 vs. 0.449 at the best fixed scale), it remains unclear whether the proposed approach offers any advantage over the existing LHSD method. A direct comparison would help clarify this point.

Regarding the response to Q2:

Thank you for the clarification. Based on the response, the main modification to the original LHSD estimator appears to be the affine calibration, while the key contribution of this paper is the identifiability and recovery theory under the tubular-noise model. I believe this theoretical contribution is valuable on its own. However, the current presentation makes it difficult to distinguish what is inherited from LHSD and what is new. I encourage the authors to make this distinction clearer and place more emphasis on the theoretical contributions throughout the paper.


Considering the points above, I have decided to maintain my original score.

Reviewer_eALPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies when noisy observations determine a unique latent manifold dimension (LMD) and at which diffusion scales score-based statistics recover it. Under bounded normal tubular noise, it establishes a reach-separation condition (2ξ0<τ02\xi_0 < \tau_0) for identifiability and gives counterexamples at the boundary. It then analyzes an affine-calibrated local Hessian spectral dimension (LHSD), deriving matching minimax rates for estimating its finite-scale field under a thin-tube condition, a fine-scale non-recovery construction, and a sharp noise–geometry bias tradeoff at intermediate scales. Synthetic experiments with DSM-trained DiTs compare LHSD with LIDL and FLIPD across diffusion scales.

Critical strengths
  1. The paper separates three questions that are often conflated: whether the latent dimension is unique, how accurately a finite-scale statistic can be estimated, and when that statistic recovers the dimension. The identifiability boundary (Theorems 2.1–2.2) and the matched-moment construction are useful; Theorem 4.1 shows that matching conditional noise means and covariances does not determine the fine-scale LHSD limit.

  2. The formal results are more precise than a qualitative account of scale sensitivity. The minimax bounds match (Theorems 3.1–3.2), and Theorem 4.3 shows that the noise–geometry bias tradeoff is tight. The experiments report unfavorable as well as favorable scales, which helps delimit where the estimator works.

Critical weaknesses
  1. The paper adds formal boundaries and rates to a familiar multiscale picture (fine scales capture noise, intermediate scales recover geometry, coarse scales oversmooth), but offers little actionable guidance. Performance is highly sensitive to the diffusion scale (in Table 1, d=100d = 100 linear LHSD RMSE jumps from 1.012 at σ=0.04\sigma = 0.04 to 104.312 at σ=0.016\sigma = 0.016), yet the theoretical scale σ2≍ξ0τ0\sigma^2 \asymp \xi_0\tau_0 depends on unknown tube width and reach, and no data-driven selection rule is provided. Deferring the related-work comparison to Appendix A also obscures what is genuinely new.

  2. Theory and implementation are disconnected. The minimax rate applies to an unrestricted empirical-DSM reference estimator, whereas the experiments use trained DiTs whose score-Jacobian errors are unmodeled. The experiments only illustrate qualitative scale dependence on synthetic data; they neither verify the theoretical sample-complexity rate nor show how to operate the estimator when dd is unknown.

Questions for authors
  1. Compared with prior multiscale and diffusion-based analyses in Appendix A (e.g., Little et al. [1], Osada [2], Martinez et al. [3]), what concrete new conclusion about noisy dimension estimation does your theory provide?

  2. Can you evaluate a data-driven rule for selecting σ\sigma (without knowing dd, ξ0\xi_0, or τ0\tau_0) on your current benchmarks and report the resulting LHSD error?

[1] Anna V Little, Yoon-Mo Jung, and Mauro Maggioni. Multiscale estimation of intrinsic dimensionality of data sets. In AAAI fall symposium: manifold learning and its applications, volume 9, pp. 04, 2009.

[2] Genki Osada. Local hessian spectral filtering for robust intrinsic dimension estimation. In Forty-third International Conference on Machine Learning, 2026.

[3] Randy Martinez, Rong Tang, and Lizhen Lin. A deep generative approach to stratified learning. arXiv preprint arXiv:2604.10650, 2026.

2 comments
AuthorsOpenReview ↗

Data-driven rule and theory for learned diffusion models

Thank you for your comments and questions. They helped us clarify the theoretical argument and establish the data-driven diffusion scale selection rule. In the revision, we address the issues you raised: The revised Introduction explains how our theory builds on earlier multiscale diagnostics. We establish when the observed law uniquely determines LMD and give sharp bounds on how closely population LHSD approximates it at intermediate scales. We also introduce a data-driven rule that selects a scale from a stable interval of the estimated LHSD curve.

W1

Section 4.4 of the revised paper gives a data-driven rule for choosing the diffusion scale without knowing the latent dimension, tube width, or reach. We adapt to LHSD the approach of Kamkari et al. [30], who applied the Kneedle algorithm of Satopaa et al. [48] to FLIPD curves. We detect bends in the median LHSD curve computed on independent selection queries, then find the widest stable interval following any detected bend and choose the grid point nearest its midpoint on the log scale. For theoretical analysis of our adaptive rule, Proposition 4.2 gives conditions on the width and location of the stable intervals and on the grid resolution under which the selected scale is of order ξ0τ0\sqrt{\xi_0\tau_0}. It then attains the optimal worst-case population bias rate among intermediate scales. Across the synthetic geometries and latent dimensions we tested, the rule generally selected scales at which LHSD accurately estimated the latent dimension on held-out queries (Table 1).

We placed the full related-work comparison in Appendix A because its scope exceeded the space available in the main text. We agree that this placement made it difficult to see what our analysis adds. The revised Introduction now discusses earlier scale diagnostics, including the multiscale singular values studied by Little et al. [38] and the gaps in the Hessian spectrum examined by Osada [43] for LHSD. We explain how our results build on these diagnostics. We identify a sharp threshold for when the observed law determines the latent dimension, quantify the worst-case bias of population LHSD across intermediate scales, and establish the minimax rate for estimating that field from a finite sample under the stated conditions.

W2

Section 4.2 shows that a trained noise predictor, including a U-Net or DiT, can match the empirical-DSM rate if its symmetrized input Jacobian is sufficiently accurate. At a fresh observation, we compare that Jacobian with the Jacobian of either the population noise predictor or unrestricted empirical DSM fitted to the same sample. If the expected squared Frobenius error relative to either reference is O(n−1σ2−d)O(n^{-1}\sigma^{2-d}) uniformly over the model class, the resulting LHSD estimator attains the minimax integrated squared-error rate O(n−1σ4−d)O(n^{-1}\sigma^{4-d}) under the conditions of Section 4.1.

Figure 10 in Appendix H.7 reports the Frobenius RMSE of the full denoiser Jacobian ID−σ∇wgI_D-\sigma\nabla_w g, normalized by D\sqrt{D}, on held-out queries across scales at n=200,000n=200{,}000. This gives a fixed-sample diagnostic related to the symmetrized noise-predictor Jacobian error in Corollary 4.1. Our bound links a small Jacobian error to accurate estimation of population LHSD and, at scales where population LHSD closely approximates the latent dimension, to accurate dimension estimation.

AuthorsOpenReview ↗

New experiments and conclusions

Q1

Our theory separates two questions that arise when a dimension estimate changes with scale. Does the observed law determine a unique latent dimension? If it does, at which scales does LHSD recover it? Little et al. [38] used local singular values across neighborhood sizes to identify scales that reveal tangent structure. Osada [43] examined whether the LHSD cutoff falls in a gap between Hessian eigenvalue clusters. Martinez et al. [40] studied local dimension on multiple strata under isotropic Gaussian observation noise. We study a single manifold and allow the tubular noise at each latent point to have any distribution within its bounded normal ball.

Section 2 shows that when the noise bound ξ0\xi_0 is less than half the reach lower bound τ0\tau_0, all admissible representations of the observed law have the same dimension. At equality, two different latent dimensions can generate the same law and hence identical diffusion scores at every scale. Even below that threshold, knowing the exact score does not guarantee that fine-scale LHSD recovers the dimension. Section 3.3 constructs models with the same manifold, latent density, and conditional noise means and covariances for which the limiting count can be any integer from dd to DD, depending on how many noise directions are locally visible. Section 3.4 gives a positive result at intermediate scales. Under its assumptions, the worst-case approximation error has order (ξ0/σ)2+(σ/τ0)2(\xi_0/\sigma)^2+(\sigma/\tau_0)^2. Balancing these terms identifies σ≍ξ0τ0\sigma\asymp\sqrt{\xi_0\tau_0} as the scale that minimizes this error up to constant factors. For finite samples, Section 4 also gives matching minimax bounds on the error of estimating the population count at a prescribed scale under its thin-tube and regularity conditions.

Q2

We evaluated the rule in Section 4.4 without using dd, ξ0\xi_0, or τ0\tau_0 to choose the scale. For each setting, we formed the median LHSD curve on independent selection queries and used Kneedle to detect bends. We chose the grid point nearest the log midpoint of the widest stable interval following a bend. Each held-out evaluation query then received its own unrounded LHSD estimate at that common scale. We used the known dd only to score those estimates.

At n=200,000n=200{,}000, the resulting dimension RMSE was below one in 14 of the 16 settings across the linear, sphere, squiggly, and Swiss roll models with d∈5,10,50,100d\in\\{5,10,50,100\\} (Table 1). On the d=100d=100 linear model you highlighted, the automatically selected scale gave an RMSE of 0.1080.108. The two errors above one were 1.0031.003 on the d=5d=5 squiggly model and 1.9241.924 on the d=50d=50 linear model. The torus entries in Table 1 use the generator's known tangent space, so we treat them as a separate diagnostic.

Reviewer_6eWAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the ID estimation of data using diffusion models when data are moved off of a proper manifold by tubular noise. The authors consider non-Gaussian ambient noise sources, specifically tubular noise, and their analysis addresses two questions: (1) does the observed distribution determine a unique latent manifold dimension? (2) At which diffusion scales can the dimension be extracted with diffusion scores?

The authors adopt a local hessian spectral dimension which is effectively the trace of a matrix A, where A = I - sigma^2 H. This is further augmented by an affine calibration following Osada [ref 42 in the paper].

The paper proves identifiability of the latent dimension under separation constraints and performs an analysis of when the LHSD recovers the LMD.

The paper concludes with a small number of experiments on simple manifold classes. The baselines are FLIPD and LIDL. At intermediate noise scales, LHSD outperforms the baselines on these manifolds. This is measured by RMSE.

This paper is difficult to parse - it is a continuous stream of dense equations, definitions, and theorems.

Critical strengths

This paper studies an important problem, intrinsic dimension estimation, and it considers an interesting case: when data are moved off the manifold by tubular noise

The identifiability proofs and analysis of when latent dimension may be retrieved may be interesting and useful for future developments in the field

The experiments, particularly those in Figure 2-3, indicate the value of the method when measuring intrinsic dimension under non-gaussian noise distributions.

Critical weaknesses

The paper is not clearly written and is very difficult to parse

The experiments are lacking: (1) there are no confidence bounds, (2) they are limited to a small number of very simple manifolds, (3) the number of diffusion scales is very limited, (4) methods such as FLIPD involve knee search to find "good" diffusion scales at which to estimate LID. This doesn't appear to have been performed.

A relevant baseline is missing from the experiments [67] Eric Yeats, Aaron Jacobson, Darryl Hannan, Yiran Jia, Timothy Doster, Henry Kvinge, and Scott Mahan. A connection between score matching and local intrinsic dimension. In NeurIPS 2025 Workshop on Structured Probabilistic Inference & Generative Modeling, 2025.

Questions for authors

Is there a practical, data-driven method to select a diffusion level at which to measure the local intrinsic dimension?

2 comments
AuthorsOpenReview ↗

Improved readability

Thank you for your constructive comments and questions. They helped us clarify the theoretical argument and strengthen the experimental evaluation. We focused the revision on the three issues you raised: We reorganized the theory to make it easier to follow, expanded the experiments to cover more geometries and diffusion scales, and introduced and evaluated a data-driven rule for choosing a scale without knowing the latent dimension.

W1

In the submitted paper, the order of the sections made it hard to see how each part of the analysis built on the previous one. Within sections, definitions, equations, and theorems often appeared in succession with too little explanation of what each result meant or why it was needed. We reorganized Sections 2–4 around three questions and added geometric explanations where the formal statements were hardest to follow.

Section 2 asks whether the observed distribution determines a unique latent manifold dimension (LMD). We moved the formal geometric assumptions to Appendix B.3 and explain the intuition in the main text. When the noise tube is narrow relative to the reach, any two candidate manifolds that generate the same distribution must lie close enough to one another to have the same dimension.

We reversed the order of the approximation and estimation analyses. Section 3 now explains when population LHSD approximates the LMD before Section 4 bounds the error in estimating that count from a finite sample. We moved the full fine-scale theorem to Appendix D. The main text explains why, at sufficiently fine scales, LHSD can count directions of variation within the noise tube. We give more space to the intermediate scales at which smoothing averages over tubular noise while preserving tangent directions. In Figure 1, the local covariance ellipse for a curve is elongated along the tangent and narrow along the normal. Section 3.1 uses this picture to explain why the normalized negative Hessian has a near-zero tangent eigenvalue that LHSD counts.

In Section 4, we now explain what the minimax rate measures before stating the bound. We fix the diffusion scale and ask how accurately a finite sample can estimate the population LHSD at that scale. We then show how this estimation error and the approximation error from Section 3 together determine the bound on LMD recovery. We placed the data-driven rule at the end of the section so that readers first see why the choice of scale matters and then how to choose one when the tube width and reach are unknown.

AuthorsOpenReview ↗

Broader experiments and data-driven scale selection

W2(1)

Appendix H.8 (Tables 2–5) now reports 95% percentile-bootstrap confidence intervals for RMSE at d∈5,10,50,100d\in\\{5,10,50,100\\}. Table 1 summarizes the point estimates across methods. We compute the intervals using 10,00010{,}000 bootstrap replicates. We resample batch mean squared errors for FLIPD, Yeats, and non-torus LHSD, and individual queries for LIDL and torus LHSD. The intervals quantify uncertainty due to the evaluation sample, conditional on the training data, the fitted model, and, for adaptive estimates, the selected scale.

W2(2)

The original subspace and sphere-based benchmarks left open how the estimators behave on folded or periodic geometries. We added a product Swiss roll and a product torus to examine these cases. The Swiss roll has a boundary, which takes it outside our theoretical assumptions. For the torus, we use the generator's known tangent space to assess how much weight the LHSD filter retains along tangent directions. Table 1 now covers five geometries and four latent dimensions (d=5,10,50,100d=5,10,50,100), for 20 settings at D=256D=256, up from nine in the submitted table.

W2(3)

The ten scales in the submission made it difficult to tell how long a low-error interval persisted between reported points. We expanded the grid from ten scales in [0.004,0.6][0.004,0.6] to 50 in [0.004,1][0.004,1] to examine both the intermediate range and the decline at larger scales. Table 1 gives five fixed-scale comparisons and the errors at the data-driven choices. For LIDL at n=200,000n=200{,}000 and d≥10d\geq10, the plotted curve interpolates five measured scales and holds the last value constant above σ=0.4\sigma=0.4. At smaller training sizes, the higher-dimensional LIDL curves are schematic. In the d=100d=100 linear panels at n=50,000n=50{,}000, 100,000100{,}000, and 200,000200{,}000, the LHSD, FLIPD, and Yeats curves show rescaled estimates and the corresponding RMSEs. Table 1 uses the original estimates. We also added training sizes of 10,00010{,}000 and 50,00050{,}000, bringing the sample-size comparison to five sizes.

W2(4)

The submitted table evaluated FLIPD at fixed scales, so it did not show the error a practitioner would obtain by running a knee search. We now report that error in Table 1. Kamkari et al. [30] applied Kneedle from Satopaa et al. [48] to the FLIPD curve. We use Kneedle to detect a bend, then select a scale from the widest stable interval that follows a detected bend. We adapted this rule to calibrated LHSD. Neither selector requires the latent dimension, tube width, or reach.

For FLIPD, we select the scale from the pooled mean curve of its evaluation batches. For LHSD, we use the median curve on independent selection queries and evaluate the selected scale on held-out queries. Under these protocols, adaptive LHSD had lower RMSE than adaptive FLIPD in 15 of the 16 settings across the linear, sphere, squiggly, and Swiss roll models. Section 4.4 gives conditions on the stable interval and the grid under which the LHSD rule attains the optimal worst-case population bias order among intermediate scales.

W3

We added the denoising-loss estimator of Yeats et al. [69], as you suggested. It estimates dimension by averaging the squared noise-prediction error over Gaussian corruptions of each query. We use the same trained DiT for this estimator, LHSD, and FLIPD, so we can compare the dimension information in the denoising residual with that in the Jacobian spectrum at the same diffusion scale. At both intermediate scales reported in Table 1, LHSD had lower RMSE than the denoising-loss estimator at every tested dimension on the linear, sphere, squiggly, and Swiss roll models. Appendix H gives the evaluation details.

Q1

As explained in W2(4), our knee plateau rule selects a scale from the estimated LHSD curve without using the latent dimension. Section 4.4 shows that, under conditions on the plateau and the scale grid, the selected scale attains the optimal order of worst-case population bias among intermediate scales, and Algorithm 1 in Appendix G specifies the full procedure.

Comments on the paper

7 comments
AuthorsOpenReview ↗

Experimental updates (2026.11.17)

We have added the results below to the revised manuscript. Only a subset of the LIDL evaluations is still running. We will add those results as soon as they are available.

Robustness to tubular noise and training sample size. Appendix H.9 reports results for 240 settings. We evaluated all five geometries (linear subspace, sphere, squiggly sphere, Swiss roll, and torus) at four training sizes (10,000, 20,000, 100,000, and 200,000), three tubular-noise ratios (0, 0.001, and 0.1), and four latent dimensions (5, 10, 50, and 100), with D=256D=256 and R=4R=4 fixed. We report dimension estimates and RMSE for LHSD, FLIPD, and the denoising-loss estimator of Yeats et al., along with the LIDL results completed so far. We also report how often the adaptive LHSD and FLIPD rules selected a scale.

Jacobian estimation error. We compared the fitted denoiser Jacobian across the four training sizes using 128 held-out queries at five diffusion scales (0.004, 0.016, 0.04, 0.16, and 0.4). For the linear model, we used the finite-scale population Jacobian as the reference. For the curved models, we computed the reference by Gaussian smoothing of the empirical training distribution. We report Frobenius RMSE for the full Jacobian, both before and after normalization by the square root of the ambient dimension.

Real-data analysis on MNIST. Appendix H.10 reports digit-specific results using the same fitted model for all ten digits. We split 256 images into 128 for scale selection and 128 for evaluation, then applied the 10% knee–plateau rule separately to each digit. The rule selected nine distinct scales, from 0.005907 for digit 8 to 0.017889 for digit 3. Digits 0 and 1 shared the same selected scale (0.007336), but their median LHSD estimates on held-out images were 90.90 and 38.91, respectively.

References.

[30] H. Kamkari, B. L. Ross, R. Hosseinzadeh, J. C. Cresswell, and G. Loaiza-Ganem. A geometric view of data complexity: Efficient local intrinsic dimension estimation with diffusion models. Advances in Neural Information Processing Systems, 37, 2024.

[48] V. Satopaa, J. Albrecht, D. Irwin, and B. Raghavan. Finding a “Kneedle” in a haystack: Detecting knee points in system behavior. International Conference on Distributed Computing Systems Workshops, 2011.

[69] E. Yeats, A. Jacobson, D. Hannan, Y. Jia, T. Doster, H. Kvinge, and S. Mahan. A connection between score matching and local intrinsic dimension. NeurIPS Workshop on Structured Probabilistic Inference & Generative Modeling, 2025.

AuthorsOpenReview ↗

Experimental updates (2026.11.17)

We have added the results below to the revised manuscript. Only a subset of the LIDL evaluations is still running. We will add those results as soon as they are available.

Robustness to tubular noise and training sample size. Appendix H.9 reports results for 240 settings. We evaluated all five geometries (linear subspace, sphere, squiggly sphere, Swiss roll, and torus) at four training sizes (10,000, 20,000, 100,000, and 200,000), three tubular-noise ratios (0, 0.001, and 0.1), and four latent dimensions (5, 10, 50, and 100), with D=256D=256 and R=4R=4 fixed. We report dimension estimates and RMSE for LHSD, FLIPD, and the denoising-loss estimator of Yeats et al., along with the LIDL results completed so far. We also report how often the adaptive LHSD and FLIPD rules selected a scale.

Jacobian estimation error. We compared the fitted denoiser Jacobian across the four training sizes using 128 held-out queries at five diffusion scales (0.004, 0.016, 0.04, 0.16, and 0.4). For the linear model, we used the finite-scale population Jacobian as the reference. For the curved models, we computed the reference by Gaussian smoothing of the empirical training distribution. We report Frobenius RMSE for the full Jacobian, both before and after normalization by the square root of the ambient dimension.

Real-data analysis on MNIST. Appendix H.10 reports digit-specific results using the same fitted model for all ten digits. We split 256 images into 128 for scale selection and 128 for evaluation, then applied the 10% knee–plateau rule separately to each digit. The rule selected nine distinct scales, from 0.005907 for digit 8 to 0.017889 for digit 3. Digits 0 and 1 shared the same selected scale (0.007336), but their median LHSD estimates on held-out images were 90.90 and 38.91, respectively.

References.

[30] H. Kamkari, B. L. Ross, R. Hosseinzadeh, J. C. Cresswell, and G. Loaiza-Ganem. A geometric view of data complexity: Efficient local intrinsic dimension estimation with diffusion models. Advances in Neural Information Processing Systems, 37, 2024.

[48] V. Satopaa, J. Albrecht, D. Irwin, and B. Raghavan. Finding a “Kneedle” in a haystack: Detecting knee points in system behavior. International Conference on Distributed Computing Systems Workshops, 2011.

[69] E. Yeats, A. Jacobson, D. Hannan, Y. Jia, T. Doster, H. Kvinge, and S. Mahan. A connection between score matching and local intrinsic dimension. NeurIPS Workshop on Structured Probabilistic Inference & Generative Modeling, 2025.

AuthorsOpenReview ↗

Experimental updates (2026.11.17)

We have added the results below to the revised manuscript. Only a subset of the LIDL evaluations is still running. We will add those results as soon as they are available.

Robustness to tubular noise and training sample size. Appendix H.9 reports results for 240 settings. We evaluated all five geometries (linear subspace, sphere, squiggly sphere, Swiss roll, and torus) at four training sizes (10,000, 20,000, 100,000, and 200,000), three tubular-noise ratios (0, 0.001, and 0.1), and four latent dimensions (5, 10, 50, and 100), with D=256D=256 and R=4R=4 fixed. We report dimension estimates and RMSE for LHSD, FLIPD, and the denoising-loss estimator of Yeats et al., along with the LIDL results completed so far. We also report how often the adaptive LHSD and FLIPD rules selected a scale.

Jacobian estimation error. We compared the fitted denoiser Jacobian across the four training sizes using 128 held-out queries at five diffusion scales (0.004, 0.016, 0.04, 0.16, and 0.4). For the linear model, we used the finite-scale population Jacobian as the reference. For the curved models, we computed the reference by Gaussian smoothing of the empirical training distribution. We report Frobenius RMSE for the full Jacobian, both before and after normalization by the square root of the ambient dimension.

Real-data analysis on MNIST. Appendix H.10 reports digit-specific results using the same fitted model for all ten digits. We split 256 images into 128 for scale selection and 128 for evaluation, then applied the 10% knee–plateau rule separately to each digit. The rule selected nine distinct scales, from 0.005907 for digit 8 to 0.017889 for digit 3. Digits 0 and 1 shared the same selected scale (0.007336), but their median LHSD estimates on held-out images were 90.90 and 38.91, respectively.

References.

[30] H. Kamkari, B. L. Ross, R. Hosseinzadeh, J. C. Cresswell, and G. Loaiza-Ganem. A geometric view of data complexity: Efficient local intrinsic dimension estimation with diffusion models. Advances in Neural Information Processing Systems, 37, 2024.

[48] V. Satopaa, J. Albrecht, D. Irwin, and B. Raghavan. Finding a “Kneedle” in a haystack: Detecting knee points in system behavior. International Conference on Distributed Computing Systems Workshops, 2011.

[69] E. Yeats, A. Jacobson, D. Hannan, Y. Jia, T. Doster, H. Kvinge, and S. Mahan. A connection between score matching and local intrinsic dimension. NeurIPS Workshop on Structured Probabilistic Inference & Generative Modeling, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.