-
I found this paper is not well written. The organization does not follow the standard structure and is hard to follow. Why is the related work a subsection of introduction? Section 2 mixes different kinds of material: Section 2.1 defines the estimator; Sections 2.2 and 2.3 are theory derivations.
-
In section 2.3, the authors explain why VAE does not overfit from a local PPCA perspective. I think the authors regard this as an important part of the contribution. However, Dai et al. 2018 has already proved this. Specifically, if the deocder mean is affine, the decoder variance is λI and the encoder is arbitrary complex, VAE collapse to PPCA.
-
The experiments are applied on synthetic or toy datasets. In addition, the paper compares only against generic intrinsic-dimension estimators. Table 1 benchmarks CorrInt, ABID, MLE, TwoNN and Local PCA, all of which operate on the raw observations or on latent means; none of them is a VAE-based estimator. The prior VAE-based approaches that the paper itself surveys in Section 1.1 — and on which its own Section 2.3 derivation directly builds — are never evaluated numerically anywhere in the paper (Zheng et al. 2022, Rolinek et al. 2019, Kumar and Poole 2020)
-
The authors claim that ``We introduced a framework for extracting manifold intrinsic dimension from trained VAEs, explained how VAE objective is conducive to ID estimation, and demonstrated its utility in rodent extracellular population recordings and bird song spectrograms." This is an over claim. In Dai et al. 2018, they proved that a nonlinear VAE can identify the number of intrinsic dimension by simply counting the number of nonzero columns in the weight of the first decoder layer (see table 1 and figure 2 of the paper). The encoder covariance statistics can also be used to identify the intrinsic dimension easily (see figure 3 of the paper).
Bin Dai, Yu Wang, John Aston, Gang Hua, and David Wipf. Connections with robust pca and the role of emergent sparsity in variational autoencoder models. Journal of Machine Learning Research, 19(41):1–42, 2018.
Yijia Zheng, Tong He, Yixuan Qiu, and David P Wipf. Learning manifold dimensions with conditional variational autoencoders. Advances in Neural Information Processing Systems, 35:34709–34721, 2022.
Michal Rolinek, Dominik Zietlow, and Georg Martius. Variational autoencoders pursue PCA directions (by accident). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12406–12415, 2019.
Abhishek Kumar and Ben Poole. On implicit regularization in β-VAEs. In International Conference on Machine Learning, pp. 5480–5490. PMLR, 2020
We thank the reviewer for acknowledging the usefulness of our method to the neuroscience community. Here we address the critiques.
1. The estimated dimension is not stable across similarly predictive architectures.
We added Appendix A.8, which explains why the Jacobian effective rank depends on network architecture: greater depth penalizes extra Jacobian dimensions more strongly but risks posterior collapse. Based on this, we propose a practical criterion for selecting the depth, and show that held-out ELBO cannot serve this purpose (Figure A.6A); see point 4 for details. Thus, the noted instability is a property of the VAE that can be dealt with in practice.
2. The exact-rank argument does not strictly apply to the effective rank.
We agree; we acknowledged this before and now do so with more emphasis. The invariance of the exact Jacobian rank does not imply invariance of the proposed effective-rank estimator. Our goal was to highlight one way in which the VAE's usefulness for ID estimation is less affected by its known shortcomings. It is not a proof of the reliability of the method, which is most convincingly shown by the empirical experiments.
3. The local Gaussian analysis explains a regularization mechanism, but does not establish consistent dimension recovery for the nonlinear Poisson model.
We agree that the local Gaussian analysis is not a proof of consistent dimension recovery for nonlinear Poisson models. Its purpose is to explain the regularization mechanism underlying VAE-ID, while the accuracy of dimension recovery is established empirically through our neuroscience-inspired simulations with known ground-truth dimensions. We have clarified this distinction in the revised manuscript (end of Section 3.2).
4. Dependence on the neighborhood choice and the 95% threshold.
We agree that every ID estimator, including ours, depends on experimenter-selected choices. Prior VAE-based methods are no exception: Zheng (2022), for instance, defines active dimensions as those with posterior variance below 0.05. We have therefore toned down our criticism that neighborhood-based methods need an arbitrary neighborhood. The real question is not whether a method has such choices, but whether they can be made reliably.
Summary. We appreciate the reviewer's careful examination of the theoretical assumptions and methodological reliability of VAE-ID. In response, we have substantially expanded our understanding of when the estimator works and what determines its accuracy. In particular, our new analysis of network depth and posterior collapse explains why similarly predictive architectures can produce different ID estimates, and motivates a practical criterion for architecture selection. We have also removed the previous neighborhood-based correction, examined threshold sensitivity, and clarified the scope of our theoretical arguments. The revised manuscript therefore offers a more complete methodology for reliable ID estimation, together with a clearer account of its theoretical foundations and limitations. We believe these advances are particularly important for neuroscience, where dimensionality estimates are increasingly used to draw scientific conclusions about neural representations, calling for practically reliable, well-tested methods. We hope our additions directly address the reviewer's primary concerns and that they warrant reconsideration of the original assessment.