acceptodds
Under review as a conference paper at ICLR 2027

Geometry-Aware Diffusion through Anisotropic Perturbations

Abstract

Score-based generative models perform remarkably well on data supported on lower-dimensional manifolds, despite relying on ambient forward processes that inject isotropic Gaussian noise in all directions. However, as the noise level vanishes, the score develops a singular normal component near the manifold, destabilizing score matching and hence complicating distribution estimation. We study a geometry-aware procedure that first perturbs data with Gaussian noise in the normal directions and then applies a standard ambient Ornstein-Uhlenbeck forward process, which regularizes the normal score without obscuring intrinsic density information. We establish non-asymptotic score estimation and distributional guarantees under -Holder integral probability metrics for . Under suitable geometric and smoothness regularity conditions, the resulting rates depend on the intrinsic dimension and match the corresponding minimax rate up to logarithmic factors, yielding better rates in the small regime compared to the usual isotropic diffusion model. Empirical evaluations on synthetic manifolds and molecular dynamics validate our theoretical framework, demonstrating the practical advantages of geometry-aware perturbations.

Primary area:
generative models
Keywords:
diffusion models, manifold learning, data perturbation, Holder integral probability metrics

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_AbeYOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies geometry aware score-based generative modelling. A method that first perturbs the data in the normal direction to the manifold, then applies the standard OU process is proposed and analysed. Minimax rate up to logarithmic factor is established for γ\gamma-Holder integral probability metrics for 0<γ≤10<\gamma\leq 1. Experimental validation is consistent with geometry aware perturbation being useful.

Critical strengths
  • Nice result showing that geometric aware perturbation which is anistropic rather than isotropic obtains minimax rate up to log factor. As far as I can tell, this is novel.
  • Practical variant of the method shows advantage on initial synthetic and a real world dataset. The advantage is not large compared to just using projection (as has been done in practice), but could still potentially be practically useful. (Update after asking AI for a review: I would down-weight this as the AI review pointed out that normal-direction perturbation followed by ambient generative modeling and projection already appears in Keegan and Ruthotto, although manifold is known in this case.)
Critical weaknesses

No critical weaknesses that I am aware of but writing should be improved, e.g. Assumption A1 is not defined but is used in the main paper (minimally a summary of what the assumption is should be provided before use, with pointer to the appendix for the full details).

Questions for authors
  • What would the implication on the results be if the intrinsic dimension d is not known?
  • Would running more reverse sampling steps for smaller γ\gamma be helpful in the experiments?
  • (Update after AI review) How is the practical method used in the experiment different from the closest practical geometry-aware diffusion methods from prior works?
3 comments

We thank the reviewer for their careful reading and assessment of the paper. We have revised the paper with a focus on the presentation, including care around stating the assumptions with reference to the appendix. We have also clarified that we study the normal perturbation method and not claim methodological novelty. We address the questions below.

Question 1: Our guarantees assume that the intrinsic dimension dd is known. If dd is unknown, a possible extension is to estimate it from the geometry-estimation sample and use d^\widehat d in the reconstruction and other dimension-dependent choices. On the event d^=d\widehat d=d, this procedure agrees with the known-dimension estimator. Provided that the output remains supported in a fixed bounded set, including on failure events, dimension misclassification contributes at most a constant times P(d^≠d)\mathbb P(\widehat d\neq d) to the expected dγd_\gamma error. Thus, a uniform O(n−1/2)O(n^{-1/2}) bound on this probability is sufficient to preserve our rates.

Such rapid dimension estimation is possible under suitable assumptions: Kim, Rinaldo, and Wasserman (2019) establish error probabilities of order O(n−cn)O(n^{-cn}), for some c>0c>0, in a noiseless manifold model. This suggests that estimating the dimension should not worsen the statistical rate. Incorporating dimension estimation into our theorem would require an estimator with a sufficiently small dimension-misclassification probability under our assumptions, together with a procedure that remains well-defined and has uniformly bounded output support when the estimated dimension is incorrect. Kim et al. provide one possible choice, subject to verifying their regularity conditions. We discuss this potential extension after Theorem 3.4.

Question 2: Thank you for this suggestion. Smaller values of γ\gamma penalize small spatial discrepancies more strongly, motivating us to examine whether additional steps improve accuracy under these metrics. Since γ\gamma specifies only the evaluation metric, we evaluate the same generated samples for every γ\gamma.

We conducted a reverse-step sensitivity study on the Swiss roll, Klein bottle, and Fourier curve added in the revision for Models B and C over three training seeds, using 500, 2000, 4000, and 8000 Euler--Maruyama steps at a fixed stopping time τ=10−3\tau=10^{-3}. We held the trained networks, perturbation scales, fitted geometry, and held-out observations fixed, and coupled Brownian paths across step counts. Each evaluation used 2000 generated samples and 2000 held-out observations per seed.

The table reports the largest absolute percentage change in mean empirical error from 500 to 8000 steps, across both models and the values of γ\gamma in each column.

DatasetAll six γ valuesSmall γ ∈ {0.05, 0.1}
Swiss roll0.205%0.007%
Klein bottle0.153%0.027%
Fourier curve1.307%0.110%

Here, the six values are γ∈\gamma\in{0.05,0.1,0.25,0.5,0.75,1}. Additional steps slightly improve the Klein-bottle results; changes on the other datasets are small and mixed. We therefore find no consistent improvement in held-out distributional accuracy or stronger benefit for smaller γ\gamma in these settings.

Question 3: Like Keegan and Ruthotto (2026), we perturb training data in normal directions, fit an ambient generative model, and project generated samples onto the manifold. The practical difference is how we obtain the geometry: they derive normals and projection from supplied geometric constraints, whereas we estimate both from observations. In the synthetic experiments, we specify a manifold family and estimate its parameters from a separate geometry sample. For alanine dipeptide, we assume no global manifold family: we use the working dimension d=2d=2 and fit local quadratic patches to Cartesian observations. In both settings, the same fitted geometry determines the perturbation directions and final projection.

Liu et al. (2025) also exploit normal–tangential geometry: Niso-DM adds normal-direction noise, while Tango-DM trains only the tangential score at small noise levels. Our construction is closely related to Niso-DM; we analyze normal perturbation followed by standard ambient diffusion and final projection, including finite-sample guarantees with estimated geometry. Models B and C share the fitted geometry and final projection, allowing us to assess the additional effect of normal perturbation.

Our principal contribution is the finite-sample statistical analysis: neural score approximation and estimation, propagation to distributional error, and minimax guarantees up to logarithmic factors, including estimated geometry under explicit reconstruction assumptions.

Reference: Kim, J., Rinaldo, A., and Wasserman, L. (2019). Minimax Rates for Estimating the Dimension of a Manifold. Journal of Computational Geometry, 10(1), 42–95.

Thanks for the answers. I retain my positive review of the paper.

Thank you for your positive assessment and thoughtful questions. Your comments helped us extend and clarify our results.

Reviewer_Y2jtOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper provides a theoretical analysis of diffusion models of low intrinsic dimension. It is hard for me to say what the main message is. Please see below for more details.

Critical strengths

None

Critical weaknesses
  1. Presentation - the paper is very hard to read. The authors use undefined jargon (see details below) and provide dry theoretical results without properly communicating why they are interesting/important or meaningful. As the authors disclose: "In this work, we used generative AI tools to assist in the writing of proofs, to provide critical ingredients for proving mathematical claims, to refine the formulation of mathematical claims, and to provide feedback on research methodology and experiment". I am not saying that the theoretical results are incorrect, or that the authors shouldn't have used AI to obtain them. What is lacking is clear communication of these results.
  2. Empirical evaluation. First, it seems like the experiments do not exactly follow the theoretical results. In authors' words: "The experimental procedures also differ from the theoretical construction". So I wonder the theoretical analysis adds value in this draft. Second, my understanding is that the main theoretical analysis focuses on adding noise in the normal direction before standard diffusion, which corresponds to model C. It seems like in all experiments, most of the gain over the standard diffusion baseline is already obtained by model B, which I understand does not rely on any of the theoretical results?
Questions for authors
  1. The term "minimax rate" is not defined in the paper.
  2. "For 0 < γ < 1, existing diffusion-based upper bounds can fail to attain the minimax rates in certain smoothness regimes under the γ-H¨older IPMs, motivating a closer examination of how geometric information can improve density estimation". Why do we need to achieve "minimax rates"? How are they related to density estimation? Diffusion models have been consistently shown to perform well on density estimation
  3. The central highlighted question in the paper: "Can utilizing or learning the geometry of the data manifold before learning the score lead to a minimax-optimal diffusion-based estimator of the distribution?" relies on an undefined term: "minimax-optimal".
  4. The introduction is not contained - it is very difficult to understand the motivation or why it is relevant without reading related work first.
  5. "With known geometry, the rate is n−(α+γ)/(2α+d) + n−1/2 up to logarithmic factors" - why is it interesting that this is the rate?
  6. "We evaluate the procedure" - which procedure?
  7. the abbreviation IPM is used three times before being defined in line 136
  8. line 225: "In this section, we show that diffusion models can achieve (near) minimax-optimality in distribution estimation under adversarial losses for 0 < γ ≤ 1 using normal perturbations, both in the known and learned geometry setting." - it has still not been explain what minimax-optimality is and why it is relevant. Also - why are adversarial losses relevant in the context of diffusion models?
  9. "the score must unravel the local geometry" - what does this mean?
  10. "The network complexity depends polynomially on the intrinsic resolution" - why is it meaningful and what is the "intrinsic resolution"? This is the only place in the paper this term is used
  11. "The recovery rate ultimately depends on how the reverse process propagates through this perturbation and how the final projection map collapses the residual ambient error." - what is the "recovery rate"?
  12. The formulation of theorem 3.4 depends on multiple results from the appendix so it's hard to process. What is the significance of this result? Why is it meaningful that these bounds are what they are?
  13. "At each geometry point" - what is a geometry point?
  14. "β(t) linear from 0.1 to 20" - I believe the authors mentioned multipled times that beta is usually constant and therefore satisfies some assumptions made?
  15. How is the alanine dipeptide trajectory sampled? How was it ensured that the heldout trajectory frames are independent of the training ones?
  16. "Explicit geometric reconstruction replaces the term n−βγ/(2α+d) in the upper bound of Tang & Yang (2024) by n−βγ/d." - why is this meaningful?
  17. "The experimental procedures also differ from the theoretical construction" - why can't the experimental procedures match the theoretical construction?
7 comments

We thank the reviewer for the careful reading. We organize our response into two comments: theoretical results and presentation, followed by experimental questions.

W1, Q4 (presentation). We revised the paper with a focus on presentation. The introduction is now self-contained: it introduces the γ\gamma-Hölder integral probability metrics (IPMs), explains minimax optimality, states the gap between existing diffusion bounds and the minimax rate, and describes the procedure we study (normal perturbation, standard ambient diffusion training, and final projection) before related work. It also explains the geometric intuition: normal perturbation smooths the manifold-supported distribution and regularizes the score in the normal directions, while nearest-point projection reverses perturbations that remain within a tubular neighborhood.

Q1, Q3 (minimax rate and optimality). Definition 2.3 defines the minimax risk Rn(P,dγ)=inf⁡p^nsup⁡p∈PE dγ(p^n,p)R_n(\mathcal P,d_\gamma)=\inf_{\hat p_n}\sup_{p\in\mathcal P}\mathbb E\,d_\gamma(\hat p_n,p), the smallest worst-case expected error achievable by any estimator from nn observations. The minimax rate is the order at which RnR_n decreases with nn. An estimator is minimax-optimal if its worst-case error attains this rate up to constants, and minimax-optimal up to logarithmic factors if it does so up to powers of log⁡n\log n. The introduction explains this informally before posing the highlighted question.

Q2, Q5 (why minimax rates matter). Strong empirical performance does not establish statistical efficiency: a method can perform well on particular datasets without a guarantee that its worst-case error decreases at the optimal rate. Matching the minimax rate certifies that no estimator can do better uniformly over the stated class. Our known-geometry bound n−(α+γ)/(2α+d)+n−1/2n^{-(\alpha+\gamma)/(2\alpha+d)}+n^{-1/2} does so up to logarithmic factors. Our use of "density estimation" was imprecise; we meant estimating the distribution under dγd_\gamma and now write "distribution estimation".

Q7, Q8 (IPMs and adversarial losses). "Integral probability metric" is now spelled out at first use. The γ\gamma-Hölder IPM has an adversarial interpretation: it is the largest difference in expectations over γ\gamma-Hölder test functions, which play the role of a discriminator class. "Adversarial" describes the evaluation metric, while training uses denoising score matching. These metrics suit manifold-supported targets because they quantify geometric error between distributions with different supports. An ambient distribution with a Lebesgue density and a lower-dimensional manifold-supported target are mutually singular, so KL is infinite and TV is maximal.

Q6, Q9–Q11, Q13 (terminology).

  • Q6: the procedure is normal perturbation, followed by standard ambient diffusion and projection at the end of the reverse process; now stated.
  • Q9: We have rewritten this in Section 3.2. At times t≪η2t\ll\eta^2, the variance in the perturbation directions remains of order η2\eta^2, while the variance along the known or estimated tangent directions satisfies σt2≍t\sigma_t^2\asymp t. Score approximation must therefore account for these distinct scales and the local geometry, motivating the small-time case of Theorem 3.3.
  • Q10: we removed the term "intrinsic resolution". At the nonparametric bandwidth h=n−1/(2α+d)h=n^{-1/(2\alpha+d)}, the number of cells of width hh in local dd-dimensional charts is of order h−d=nd/(2α+d)h^{-d}=n^{d/(2\alpha+d)}. The small-time network width and sparsity in Theorem 3.3 scale with this quantity up to logarithmic factors, so their sample-size exponents depend on the intrinsic dimension dd and not on the ambient dimension DD. This does not, however, exclude ambient-dimension dependence in constants.
  • Q11: by “recovery rate,” we meant the convergence rate of the expected distribution-estimation error. Theorem 3.4 is renamed to "Distribution Estimation"
  • Q13: these are the observations in the geometry-estimation sample; we have clarified this in the revision.

Q12, Q16 (significance of Theorem 3.4). The assumptions and the reconstruction are now described before the theorem, which can be read without consulting the appendix. Its significance is that the estimator attains the minimax rate up to logarithmic factors. Each term is a term of the minimax benchmark: n−1/2n^{-1/2} is the parametric term, n−(α+γ)/(2α+d)n^{-(\alpha+\gamma)/(2\alpha+d)} is the cost of estimating an α\alpha-smooth density on a dd-dimensional manifold, and n−βγ/dn^{-\beta\gamma/d} is the cost of recovering the support. The upper bound of Tang and Yang (2024) instead contains n−βγ/(2α+d)n^{-\beta\gamma/(2\alpha+d)}. For d≥2d\geq2, replacing this term with n−βγ/dn^{-\beta\gamma/d} improves the polynomial rate exactly when γ(β−1)<α\gamma(\beta-1)<\alpha. This sharpens the available guarantee; it does not establish that isotropic diffusion necessarily converges at the slower rate. We moved this discussion from Section 5 to directly after Theorem 3.4.

AuthorsOpenReview ↗

Response to Reviewer Part 2

This comment continues our response (Part 2 of 2: experimental questions).

W2 (first point), Q17 (why the experiments differ from the theoretical construction). The main obstacle is computational. The theoretical reconstruction involves a minimum-distance search over a class of regular manifolds, which establishes the statistical upper bound but does not give an efficient algorithm; likewise, the sparse ReLU network classes are chosen for the approximation and covering arguments. The value of the analysis does not rest on the experiments: it establishes finite-sample guarantees at the minimax rate up to logarithmic factors, while the experiments assess practical implementations of the same procedure at fixed sample sizes and do not test convergence rates. We removed the quoted sentence. Section 4 now states this, and Appendix F.4 lists each substitution (network class, loss weighting, selection of η\eta and τ\tau, schedule, sampler, geometry fitting).

We also revised and reran the experiments to follow the theory's geometric construction more closely. The initial implementation used local PCA normal estimates and a local affine correction, a cheap alternative common in manifold learning. The revised implementation derives both perturbation directions and final projection from the same geometry, fitted from the geometry-estimation sample: parametric families for the synthetic data and local quadratic patches in Cartesian coordinates for alanine dipeptide. This implements the shared-geometry principle used in the theory. However, fitting a known family does not implement the general unknown-manifold reconstruction, and a collection of local patches is not guaranteed to form a globally regular manifold. We therefore distinguish these implementations from the estimator covered by the theorem and do not claim that its guarantees apply directly to the experimental results.

W2 (second point): Model B versus Model C. Comparing A and B assesses final projection, while comparing B and C assesses the additional effect of normal perturbation under the same projection. Projection corrects off-manifold deviations, but does not by itself ensure that the distribution along the manifold is accurately learned. In the revised results, Model C has the lowest mean error in every synthetic setting and for both alanine targets at every γ\gamma. While projection accounts for most of the gain on the Klein bottle and the Fourier curve, the additional improvement is comparable or larger in other settings: at γ=1\gamma=1, the reduction from B to C is at least as large as that from A to B for the sphere (D=5,10D=5,10) and the torus (D=5D=5). For the projected alanine target, mean W1W_1 decreases from .224.224 to .209.209 to .169.169 for A, B and C, compared with a reference of 0.1340.134 between two independent batches of simulated trajectory frames. These comparisons use settings selected on validation data. We report differences in mean error without claiming statistical significance. Our analysis requires positive normal perturbation and does not cover Model B: when η=0\eta=0, the normal variance also vanishes as diffusion time approaches zero, so the regularization used in our score-approximation argument is absent. Establishing statistical guarantees for standard diffusion followed by final projection remains future work.

Q14 (schedule). A constant schedule is common in theoretical analyses and is a simple example satisfying our assumptions, but it is not required; our earlier wording did not distinguish this example from the linear schedule used in the experiments. The experiments use the linear VP-SDE schedule β(t)=0.1+19.9t\beta(t)=0.1+19.9t on [0,1][0,1], which is bt=0.05+9.95tb_t=0.05+9.95t in our notation. On [0,1][0,1] this schedule is positive, bounded and smooth, so it satisfies the schedule bounds on that interval. The linear schedule satisfies the bounds on [0,1][0,1], but not the global boundedness condition in Assumption 3.2. Section 3.1 now distinguishes global bounds from finite-interval bounds, and Appendix F.4 specifies the experimental schedule and horizon.

Q15 (alanine dipeptide). In the initial submission, training and test frames were drawn at random from a single trajectory, so held-out frames could be temporally adjacent to training frames. We redid the experiment using the three independent 250 ns alanine dipeptide simulations from mdshare (trajectories 0, 1 and 2). Training frames come from the first half of trajectory 0 and geometry-estimation frames from its second half, after a 5 ns gap. Validation frames, used only to select η\eta and τ\tau, come from trajectory 1, and test frames from trajectory 2. Test data therefore come from a separate simulation. Within each split, frames are selected without replacement from a 20 ps grid, but temporal spacing need not make frames within a trajectory independent. Details are in Section 4.2 and Appendix F.3.

I thank the authors for their detailed response and updated revision.

I believe that the changes implemented by the authors make the paper more accessible and I think that the communication and interpretation of results are as important (if not more important in the age of LLMs) than the results themselves. Since the main contribution is theoretical, I wonder whether ICLR is the best venue for the presented results. For example, the results the authors are building on by Tang & Yang were published in Annals of Statistics and AISTATS, which are much more theory-focused than ICLR and to me seem more appropriate.

It is hard for me to assess whether these results are of interest to the wider community of ICLR.

Reviewer_nhMkOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Diffusion modeling of manifold-supported data is a known challenge. The paper addresses a technique in the literature where diffusion models model data that is first perturbed in the direction orthogonal to the manifold. The paper offers novel distributional guarantees and minimax theory for this approach and provides strong theoretical support to the existing technique. The proposed theory is also validated on a suite of experiments. Overall, the paper is a strong theoretical contribution to the constrained generative modeling literature, dramatically improving the statistical reliability of an existing technique in a way that is accessible and useful to the broader machine learning community. I strongly recommend this work for publication.

Critical strengths
  1. The paper offers excellent theoretical rigor and distributional guarantees to work that is already present in the literature, improving its reliability and statistical fidelity for a broader range of scientific users. The topic is also of strong relevance to the ICLR community and connections to AI alignment with scientific constraints.
  2. The paper is well-written and builds up to the proposed theory well. Theoretical results are presented naturally in a narratively cohesive manner.
Critical weaknesses
  1. The only notable difference between it and the empirical results are that Definition 2.3 requires a ReLU network but SiLU activations appear to be used in the experiments. Other than this, the paper is overall technically sound.
Questions for authors
  1. It is not entirely clear to me why Definition 2.3 is needed and why assuming a ReLU network structure is necessary.
  2. To the above point, is there any reason why a SiLU network was used instead of a ReLU network in the experiments (or how sensitive the presented theory is to the architectures assumed in Definition 2.3?
  3. Is the alanine dipeptide task executed on a real or synthetic dataset?
  4. How are the candidate perturbation scales eta selected?
  5. It may also be worth investigating and citing https://arxiv.org/abs/2105.12152v3, as their theoretical results in Section 3.3 may be relevant.
  6. The title bears substantial similarity to https://arxiv.org/abs/2601.23151, which is already cited and referenced. Perhaps a title clarifying that the contributions of the paper are evaluative or theoretical in nature would better introduce the work.
  7. Regarding the sentence "These insights motivate the question of how diffusion models, or more specifically the score function, can learn the geometry of the manifold" - some citations that may be useful for framing this question could be https://arxiv.org/abs/2607.04514, https://arxiv.org/abs/2605.20235, and https://dl.acm.org/doi/10.5555/3692070.3693712.
3 comments

We thank the reviewer for highlighting the strengths and relevance of our work and for their strong endorsement. We address the questions below.

Questions 1 and 2: Definition 2.3 (now Definition 2.4) specifies the ReLU neural-network class used to construct the theoretical score estimator. Its constraints on depth, width, sparsity, parameter bounds, and output bounds allow us to quantify both the score approximation error and the statistical complexity of the class.

We prove our theoretical guarantees for ReLU networks because established approximation results and complexity bounds make this class more convenient to analyze. This is also a common choice in existing statistical analyses of diffusion models. Practically, the normal perturbation and projection steps can be implemented with other activations. We chose SiLU as a smooth activation commonly used in diffusion implementations, following established practice in score-based generative modeling. We use the same activation across all compared configurations so that the comparisons assess the effects of normal perturbation and final projection.

Relevant theoretical results also exist for other activations. For instance, Ohn and Kim (2019) establish quantitative approximation and statistical estimation guarantees for general activation functions, including SiLU. Zhang, Lu, and Zhao (2024) establish approximation results transferring ReLU networks to other activations, including SiLU, by showing that a ReLU network can be approximately arbitrarily well with a network of the same width and depth. These results provide a starting point for extending our guarantees to SiLU. We leave this extension to future work because it requires a separate analysis of how accurately SiLU networks approximate the score across diffusion times and how the network complexity scales with the required accuracy.

Question 3: The alanine dipeptide data are simulation-generated: they come from molecular dynamics trajectories distributed through mdshare, not experimental measurements. Unlike our synthetic manifold examples, these trajectories model molecular motion and include thermal fluctuations. We have made this distinction explicit in the revised experimental section. We also clarify that the two-dimensional geometric approximation is a modeling choice motivated by the backbone dihedral angles; we do not assume that the Cartesian trajectory lies exactly on a two-dimensional manifold.

Question 4: We thank the reviewer for requesting this clarification. The choice of η=0.1\eta=0.1 was a fixed choice for the preliminary experiments. In the revised synthetic experiments, we select $\eta \in $ {0.01,0.02,0.05,0.1,0.20.01,0.02,0.05,0.1,0.2} separately for each training seed using mean empirical validation W1W_1 over eight batches. The selection is frozen before test evaluation, and the same selected model is evaluated under every reported dγd_\gamma. For alanine, we jointly select (η,τ)(\eta,\tau) using validation dγd_\gamma averaged over the four reported values of γ\gamma and two batches, with separate trajectories for validation and testing.

Question 5: We thank the reviewer for this suggestion and have added Horvat and Pfister's paper to the introduction. Their inflation–deflation approach provides an important precedent for normal perturbation before ambient density estimation with normalizing flows. Section 3.3 also highlights how tangential smoothing blurs the intrinsic density and the role of reach in recovering the original manifold point from sufficiently small normal perturbations, closely connecting to our construction. Our work establishes finite-sample score-estimation and distributional guarantees for diffusion, including estimated geometry.

Question 6: We thank the reviewer for suggesting a title with more clarity of the nature of our paper. We intend to change the title to "Geometry-aware Diffusion through Normal Perturbations: A Statistical Analysis".

Question 7: We thank the reviewer for these helpful references. We have incorporated the recommended works into the introduction: Brosse and Dalalyan on how support geometry enters small-noise expansions of the score and its derivatives; Huang et al. on learning a denoising representation followed by residual score learning; and Permenter and Yuan on interpreting denoising as approximate manifold projection. These references strengthen the motivation for using geometric information before score estimation, and we retain the complementary references on intrinsic-dimension estimation and the statistical difficulty of support recovery.

References:

Ohn, I. and Kim, Y. (2019). Smooth Function Approximation by Deep Neural Networks with General Activation Functions. Entropy, 21(7), 627.

Zhang, S., Lu, J., and Zhao, H. (2024). Deep Network Approximation: Beyond ReLU to Diverse Activation Functions. Journal of Machine Learning Research, 25(35), 1–39.

I thank the reviewers for their detailed response! My concerns are addressed. I maintain my original positive rating of this paper being a clear acceptance.

Thank you for your thoughtful review and continued support of the paper. Your suggestions helped us sharpen the motivation and positioning of our contributions.

Comments on the paper

1 comment
Area_Chair_aNPEOpenReview ↗

Reviewer-author discussion kickoff

Dear Reviewers,

Thank you for your review. The authors have now provided their rebuttal. We would appreciate it if you could take some time to read their responses and engage in the discussion, particularly on whether your concerns have been adequately addressed.

Thank you for your time and contributions to the review process!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.