- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1(−1)
Read review
Paper summary
This paper theoretically studies when the latent manifold dimension can be uniquely defined and recovered from noisy observations. It first establishes an identifiability result showing that, under bounded normal noise, the latent dimension is uniquely determined when , and also provides counterexamples at the boundary. It then analyzes the existingLID estimator, LHSD, across diffusion scales, showing that very small scales can capture noise-induced directions, while very large scales erase dimension information; at suitable intermediate scales, an affine-calibrated version of LHSD recovers the latent dimension with a sharp bias tradeoff and an optimal scale of approximately . The paper further derives matching finite-sample minimax rates for estimating the finite-scale LHSD field. These theoretical results are validated on synthetic tubular manifolds using DSM-trained DiTs, where the experiments reproduce the predicted fine-, intermediate-, and coarse-scale behavior and show accurate dimension recovery at appropriate intermediate scales.
Critical strengths
S1. Timely theoretical explanation of an important empirical phenomenon in diffusion-based dimension estimation.
Recent generative-model-based LID estimators, such as LHSD and FLIPD, have shown the potential to substantially improve over traditional kNN-based estimators and have attracted increasing attention. Previous empirical studies have suggested that their estimation accuracy can vary dramatically with the diffusion noise scale, but the mechanism behind this scale dependence has remained unclear. A major strength of this paper is that it provides a theoretical explanation for this phenomenon, clarifying why different diffusion scales can lead to qualitatively different dimension estimates.
S2. Strong and coherent theoretical framework for latent-dimension identifiability and recovery.
The paper first asks the fundamental question of whether the latent manifold dimension is identifiable from noisy observations, establishing the clean condition together with a counterexample at the boundary. It then characterizes how the existing LHSD estimator behaves across diffusion scales: fine scales may capture noise-induced directions, intermediate scales recover the latent dimension, and excessive smoothing removes dimension information. The resulting sharp bias tradeoff and the population-optimal scale provide a particularly intuitive picture of when dimension recovery is possible.
S3. Substantial statistical guarantees complement the geometric analysis.
Beyond the population-level recovery results, the paper derives matching minimax rates for estimating the finite-scale LHSD field. By explicitly separating structural identifiability, approximation bias, and finite-sample estimation error, the analysis provides a fairly complete theoretical account of the different sources of difficulty in recovering latent dimension under the tubular-noise model.
Critical weaknesses
W1. From a practitioner’s perspective, the importance of diffusion-scale selection is not itself a new finding.
Prior work on FLIPD and LHSD had already documented strong sensitivity to the diffusion noise scale and proposed practical heuristics or diagnostics for selecting suitable scales. Therefore, the practical message that inappropriate scales should be avoided was already known. The main novelty of this paper is instead the theoretical explanation of this phenomenon, through the characterization of fine-, intermediate-, and coarse-scale regimes and the associated bias tradeoff. I think this distinction should be stated more clearly.
W2. There remains a substantial gap between the theoretical estimator and the learned diffusion estimator used in the experiments.
The minimax guarantees are established for an unrestricted empirical-DSM reference estimator, rather than for the DSM-trained DiT used in the experiments. This distinction is particularly important for LHSD because it relies on the score Jacobian/Hessian, and accurate score estimation does not automatically imply accurate derivative estimation. Moreover, the empirical validation is restricted to synthetic tubular-manifold models, and the curved-model references are local-tube approximations rather than exact population LHSD values. Thus, it remains unclear to what extent the theoretical guarantees carry over to practical learned diffusion models and real-world data.
W3. The attribution and novelty boundary surrounding LHSD are insufficiently clear.
LHSD is an existing method introduced in prior work [42], and the paper itself states in the introduction that it “adopt[s] local Hessian spectral dimension (LHSD) [42].” However, the abstract and Section 3 use language suggesting that LHSD is being introduced in the present paper, and the affine-calibrated variant is still referred to simply as “LHSD” in the experiments rather than being clearly distinguished from the original method. The actual methodological novelty appears to be the affine calibration together with the new identifiability, scale-dependent recovery, and minimax analyses. Among the concerns above, I view this as the most serious one, because it directly affects the attribution and novelty of the central method rather than only its practical applicability or empirical validation. The distinction between the existing LHSD method and the contributions of this paper should therefore be made explicit throughout the manuscript.
[42] Genki Osada. Local hessian spectral filtering for robust intrinsic dimension estimation. InForty-third International Conference on Machine Learning, 2026.
Questions for authors
Q1. Can the authors connect the theoretical oracle scale to an observable, data-driven scale-selection criterion?
In particular, could the existing scale-selection diagnostics used in LHSD, or related heuristics in prior diffusion-based LID methods, be theoretically justified under the tubular-noise model, or at least empirically shown to select scales close to the theoretically favorable intermediate regime? Such a result would substantially strengthen the practical relevance of the theory.
Q2. Could the authors clarify precisely which components of LHSD are inherited from [42] and which are new in this paper?
In particular, is the only modification to the existing LHSD estimator the affine calibration, with the main new contributions being the identifiability and scale-dependent recovery theory? If so, I believe the terminology and novelty claims should be revised consistently throughout the abstract, method section, and experiments.
[42] Genki Osada. Local hessian spectral filtering for robust intrinsic dimension estimation. InForty-third International Conference on Machine Learning, 2026.
11 comments
Attribution, data-driven selection rule, and affine calibration
W3
We agree that the submitted abstract obscured that attribution. The revised abstract credits Osada at the first mention of LHSD, and the Introduction describes the original estimator and its scale diagnostic. Section 3.2 presents the original spectral filter before defining our affine calibration, the only change we make to the count at a fixed scale. We specify there that subsequent uses of LHSD refer to the calibrated count, and Section 5 identifies the estimator used in the experiments as calibrated LHSD.
We also revised the statement of our theoretical contributions. After establishing when the observed law identifies the latent dimension in Section 2, we treat the two errors in recovery in order. Section 3 bounds the approximation error of population LHSD as a function of diffusion scale. Section 4 then gives the minimax rate for the estimation error when that population field is estimated from finite samples at a prescribed scale. The revision adds a sufficient condition under which a plug-in estimator based on any differentiable noise predictor attains this rate, expressed in terms of the accuracy of its input Jacobian (Section 4.2). We also added a data-driven rule for choosing the scale and showed when it attains the optimal worst-case population bias order among intermediate scales (Section 4.4).
Q1
Kamkari et al. [30] used Kneedle (Satopaa et al. [48]) to find bends in FLIPD curves. In Section 4.4, we apply it to the median LHSD curve computed on independent selection queries. We find the widest stable interval after any detected bend and choose the grid point closest to its midpoint on the log scale. This choice requires no knowledge of the latent dimension , the tube width , or the reach bound .
Proposition 4.2 explains when this choice works. It requires a sufficiently wide stable interval, with all competing intervals of at least that width lying in the specified scale window and a grid fine enough to resolve their log midpoints. Under the stated model conditions, for sufficiently small , the selected scale is of order and attains the optimal worst-case population bias rate among intermediate scales. The two-shell example in Appendix G shows that even with exact population counts and the balancing scale on the grid, a stable interval alone cannot guarantee near-optimal bias. Whether the spectral-gap diagnostic of Osada [43] admits a similar guarantee remains open.
On held-out queries, adaptive LHSD achieved an RMSE below one in 14 of the 16 settings across the linear, sphere, squiggly, and Swiss roll models (Table 1). On the linear model, however, RMSE was at the selected scale and at the best of the five reported fixed scales. In Figure 4, we will report the gap between RMSE at the selected scale and the lowest RMSE on the full evaluation grid, both computed from the original, unscaled estimates. We use the known dimension only to evaluate the choice.
Q2
Yes. At a fixed scale, we use the same Hessian eigenvalues and fourth-order filter as Osada [43]. We still sum the filter weights, but rescale each weight before taking the sum. For an ideal normal projector, the original filter gives each tangent direction weight one and each normal direction weight . The rescaling sends these weights to one and zero, respectively, so the calibrated count equals . For the same Jacobian, the two counts differ by at most , which is for and .
The data-driven rule chooses the scale at which to evaluate this count. Our theory addresses when the observed law identifies the latent dimension, how closely the population count approximates it, and how accurately the count can be estimated from a finite sample.
Regarding the response to Q1:
The original LHSD paper already proposed a transition-mass diagnostic for scale selection, but the authors do not compare their Kneedle-based approach against it. Given the relatively large error in the (d=50) setting (RMSE 1.924 vs. 0.449 at the best fixed scale), it remains unclear whether the proposed approach offers any advantage over the existing LHSD method. A direct comparison would help clarify this point.
Regarding the response to Q2:
Thank you for the clarification. Based on the response, the main modification to the original LHSD estimator appears to be the affine calibration, while the key contribution of this paper is the identifiability and recovery theory under the tubular-noise model. I believe this theoretical contribution is valuable on its own. However, the current presentation makes it difficult to distinguish what is inherited from LHSD and what is new. I encourage the authors to make this distinction clearer and place more emphasis on the theoretical contributions throughout the paper.
Considering the points above, I have decided to maintain my original score.
On practical perspective and theory for learned diffusion models
Thank you for pointing out where our wording obscured the contribution. In particular, the submitted abstract used introduce for a statistic developed by Osada [43]. We corrected that attribution throughout the paper. We also revised Section 3 to explain what our theory adds to existing scale diagnostics and Section 4 to state precisely when the empirical-DSM guarantee extends to a learned predictor. Section 5 now evaluates the dimension error at a scale selected from data.
W1
We agree with your assessment. Practitioners already had guidance on scale selection. Kamkari et al. [30] used a knee in the FLIPD curve, and Osada [43] examined whether the LHSD cutoff lies in a gap between Hessian eigenvalue clusters. The revised Introduction credits both approaches and makes our theoretical contribution explicit. Section 3.4 gives matching upper and lower bounds on the worst-case error with which population LHSD approximates the latent dimension at intermediate scales. Under its assumptions, these bounds identify the scale that balances tubular noise and curvature and minimizes this error up to constant factors. This explains why the choice of scale matters even when the score is known exactly.
An important update is that we also added a data-driven diffusion scale selection rule. Section 4.4 gives conditions on the width and location of the interval and on the grid resolution under which the rule selects a scale of order ξ0τ0 and attains the optimal worst-case population bias order among intermediate scales. In Table 1, adaptive LHSD had lower dimension RMSE than adaptive FLIPD in 15 of 16 settings across the linear, sphere, squiggly, and Swiss roll models under their respective selection protocols. This gives practitioners a way to choose a scale without knowing the tube width or reach, along with an explanation of when that choice is justified.
W2
We agree that the empirical-DSM theorem alone does not establish a rate for the DiT used in our experiments. A model can predict noise accurately at corrupted inputs and still have inaccurate derivatives at the observed queries where we compute LHSD. Section 4.2 now states what would suffice. At a fresh observation, we compare the fitted predictor's symmetrized input Jacobian with that of either the population predictor or unrestricted empirical DSM fitted to the same sample. If the expected squared Frobenius error relative to either reference is O(n−1σ2−d) uniformly over the model class, the learned LHSD estimator attains the minimax integrated squared-error rate O(n−1σ4−d) under the conditions of Section 4.1. This condition applies to any differentiable noise predictor, including a DiT or a U-Net.
Figure 10 in Appendix H.7 reports the Frobenius RMSE of the full denoiser Jacobian ID−σ∇wg, normalized by D, on held-out queries across scales at n=200,000. It uses a population reference for the linear model and an empirical-DSM reference for the curved models. We plan to repeat the comparison across training sizes to see how the discrepancy changes with n. The bound explains why small Jacobian error can lead to accurate dimension estimates at scales where population LHSD closely approximates the latent dimension.
Your concern about the curved-model references matters when interpreting the experiments. For the linear model, we use a numerical population LHSD reference. For the sphere and its radial deformation, the local-tube reference is itself an approximation, so its discrepancy from learned LHSD cannot be attributed entirely to the DiT. The known latent dimensions in these synthetic benchmarks let us measure dimension error directly in Table 1. On the MNIST dataset, the latent dimensions are unknown. Our planned experiments will therefore report how stable the estimates are across scales and how they vary across image classes.