- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies what pooled TwoNN measures in high-dimensional finite-sample regimes. For anisotropic Gaussian data, the authors show that TwoNN concentrates around an explicit sample-size- and covariance-spectrum-dependent effective dimension, rather than necessarily recovering the population dimension. They further derive a corresponding sample-resolution law for uniform data on a noiseless sphere and show that polynomially many samples recover only a vanishing fraction of the true dimension as the dimension grows.
The paper validates these phase laws experimentally on Gaussian and spherical data, studies finite-sample target estimation, and examines neural representations from CIFAR-10 ResNet-18 models. In the neural experiments, function-preserving coordinate transformations can substantially change TwoNN while leaving predictions unchanged, and fitted Gaussian and class-Gaussian-mixture null models do not fully account for the observed TwoNN values. Overall, the paper argues that TwoNN should be interpreted as a sample-resolution-dependent quantity whose value depends on the representation geometry and metric, rather than as a direct estimate of population manifold dimension.
Critical strengths
S1. The paper provides an explicit characterization of what TwoNN measures in high-dimensional, finite-sample regimes.
Rather than merely pointing out finite-sample bias, it shows that pooled TwoNN converges, for Gaussian data, to a sample-size- and spectrum-dependent effective dimension . This gives a substantially more precise interpretation of the estimator.
S2. The paper clearly distinguishes population manifold dimension from sample-resolution-dependent effective dimension.
The noiseless-sphere result is particularly compelling: even when the true manifold dimension is known exactly, polynomially many samples may allow TwoNN to recover only a vanishing fraction of it in high dimensions. This provides a clean theoretical demonstration of the resolution limits of nearest-neighbor-based ID estimation.
S3. The theoretical results are supported by careful finite-sample experiments.
The Gaussian and spherical experiments test the proposed phase laws directly, while the later-neighbor analysis and neighbor-window correction help separate resolution effects from approximation and target-estimation errors.
S4. The neural representation experiments provide a strong practical stress test of the usual interpretation of TwoNN.
By applying invertible, function-preserving coordinate transformations and compensating the classifier accordingly, the paper shows that TwoNN can change by roughly a factor of two while all predictions remain unchanged. This makes the dependence of TwoNN on representation geometry and metric structure especially concrete.
Critical weaknesses
W1. The main theoretical results do not directly cover the low-intrinsic-rank regime that is often of primary interest in manifold-based LID estimation.
In the Gaussian analysis, the assumptions imply that the covariance rank grows as , while fixed-rank or sublinear-rank settings are explicitly outside the asymptotic regime considered. This leaves some uncertainty about how the proposed sample-resolution law extends to the more classical setting in which data lie near a manifold of dimension . Clarifying whether the same qualitative phenomenon is expected in such low-rank or noisy-manifold regimes would help establish the broader scope of the theory.
W2. The neural coordinate-intervention experiment is informative, but its interpretation depends on what invariances an intrinsic-dimension estimator is expected to satisfy.
The invertible transformations preserve the network function and the information contained in the representation, but they also change the Euclidean metric geometry on which TwoNN is based. Therefore, a substantial change in TwoNN under such transformations is not entirely surprising for a distance-based estimator. The result is still a useful stress test, but its implications would be clearer if the paper more explicitly discussed which classes of coordinate transformations an intrinsic-dimension estimate should ideally be invariant to. In addition, the neural experiments are limited to CIFAR-10, ResNet-18, five trained models, and a specific intervention site, so the generality of the observed effect remains open.
Questions for authors
Q1 (related to W1).
Do the authors expect the sample-resolution phenomenon identified in this paper to extend to low-intrinsic-rank or noisy-manifold settings with ? A brief discussion of this regime would help clarify the scope of the theory.
Q2 (related to W2).
What class of coordinate transformations do the authors believe an intrinsic-dimension estimator should ideally be invariant to? Clarifying this point would help interpret the function-preserving coordinate-intervention experiment.
Q3.
The paper interprets TwoNN as measuring a sample-resolution-dependent finite-radius dimension rather than the population manifold dimension. This seems closely related to recent density-/diffusion-based LID estimators such as LHSD [1] and LIDL [2], where the effective resolution is instead controlled by a density-estimation or smoothing/noise scale. Could the authors discuss how these two lines of work are related, possibly in the Related Work or Discussion section? In particular, is the exponential sample requirement derived here specific to nearest-neighbor estimators, or should one expect an analogous resolution--sample-complexity tradeoff for density-/diffusion-based LID estimation?
[1] Osada, Genki. "Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation." ICML 2026. [2] Tempczyk, Piotr, et al. "Lidl: Local intrinsic dimension estimation using approximate likelihood." ICML 2022.
2 comments
Thank you for your detailed response. The strengths of this paper, as noted in my original review, remain clear. I also appreciate the clarification of the connection between TwoNN and LIDL through their shared dependence on radial probability mass.
Given the growing interest in density- and diffusion-based LID estimators, I believe that advancing the theoretical understanding of a classical estimator like TwoNN is a valuable contribution that merits acceptance. I therefore maintain my positive assessment.
Low Rank, Coordinate Invariance, and Density-Based Estimation
We thank the reviewer for their positive assessment and the questions about the scope of the results. We use the review's W1/W2 and Q1/Q2/Q3 labels.
1. Low-rank Gaussians and noisy manifolds (W1, Q1)
The Gaussian theorem also covers data whose intrinsic rank is much smaller than ambient dimension, provided that rank grows. For a Gaussian supported on an r-dimensional subspace of RD, orthonormal coordinates on that subspace preserve pairwise distances and TwoNN values. If r→∞ and the restricted covariance satisfies Theorem 3.1's assumptions, the sample-resolution law applies with logn/r, even when r≪D. Thus, growing low-rank Gaussian data are covered by applying the theorem in the dimension of their support.
For curved or noisy manifolds, TwoNN can report different dimensions depending on the neighborhoods reached by the sample. Equation (5) relates neighbor ratios to probability-mass growth over those neighborhoods. When the sampled radii are too large to resolve the local manifold, TwoNN may report a different dimension. Noise can also change mass growth at small radii by introducing variation normal to the manifold. Establishing corresponding convergence laws would require assumptions on curvature, density, noise, and dependence between sampled neighbors.
2. What invariance should TwoNN satisfy? (W2, Q2)
An estimator of population manifold dimension should have the same population limit under smooth coordinate changes with smooth inverses. Such transformations preserve manifold dimension, but can change Euclidean neighbor distances. TwoNN is invariant to translations, orthogonal transformations, and uniform nonzero scaling because these preserve neighbor-distance ratios. Anisotropic transformations generally alter those ratios. If distances are adjusted to account for the coordinate transformation, all pairwise distances and TwoNN estimates remain unchanged. Our experiments keep Euclidean distance after changing feature coordinates to measure how much the reported TwoNN value changes.
The additional contribution is the measured size and alignment dependence of that effect. Across five ResNet-18 models, PCA-aligned rescaling increases pooled TwoNN by 2.199-fold, whereas random orientations with the same singular values yield 0.995 (Table 2). We compensate the classifier weights with the inverse transformation, preserving the model's predictions. The original empirical-CDF implementation gives corresponding factors 2.348 and 0.994. The control isolates the effect of aligning the rescaling with covariance directions rather than changing the rescaling magnitudes. Theorem 4.1 also constructs a rise and fall in pooled TwoNN under invertible Gaussian contractions while population dimension stays fixed. A further test on one frozen ViT-Base/16 gives pooled factors 1.641 and 0.997 under PCA alignment and matched random orientations (Appendix P). The experiments quantify coordinate sensitivity in both tested architectures.
3. Relation to LIDL and LHSD (Q3)
TwoNN and the density-based methods are connected through the probability mass around a point. Let Gx(r)=P(∥Y−x∥≤r). The Gaussian-smoothed density used in LIDL can be written as
pσ(x)=(2πσ2)−D/2∫0∞e−r2/(2σ2)dGx(r).
Thus, TwoNN's neighbor ratios and LIDL's likelihood changes depend on the same radial mass distribution, but probe it differently. TwoNN evaluates mass growth around radii selected by n. LIDL varies the smoothing scale σ and estimates dimension from the resulting change in log likelihood. For local mass growth Gx(r)∝rd, the smoothed density scales as σd−D in the local regime. LHSD examines the Hessian spectrum of the smoothed log density, filtering directions associated with normal rather than tangent variation. Its statistic uses second spatial derivatives of that smoothed distribution.
This connection suggests a finite-resolution comparison across estimators. TwoNN selects radii at probability mass near 1/n, which drives its exponential threshold. Density-based methods set the smoothing scale separately, while estimating the smoothed density and its derivatives from finite samples. One could study how the mass profile, sample size, ambient dimension, and smoothing scale jointly determine the reported dimension. The Gaussian and spherical large-deviation calculations in this paper offer starting points for analyzing such scale transitions, with additional work needed for density-estimation error and Hessian estimation. We have added this connection to Related Work.