acceptodds
Under review as a conference paper at ICLR 2027

Geometry-Aware Ornstein–Uhlenbeck Diffusion via Structured Drift Design

Abstract

Diffusion-based generative models have achieved remarkable success in high-dimensional data generation; however, they fundamentally rely on isotropic diffusion processes that destroy meaningful geometric structures in the forward process. For complex, multimodal, and highly correlated distributions—such as biologically constrained genetic data—isotropic noise merges distinct modes and distorts intrinsic dependencies. This forces the reverse process to recover structure from heavily degraded signals, leading to slow convergence, mode averaging, and biologically implausible samples. To address this, we introduce the Geometry-aware Ornstein–Uhlenbeck (GOU) process, a structured drift design that embeds data geometry into the forward and backward dynamics. By employing a variance-aware anisotropic drift, GOU contracts low-variance directions rapidly while preserving high-variance directions longer, maintaining key multimodal structures as stable channels over time. We provide rigorous theoretical analysis of the proposed dynamics, including the evolution of moments in the forward process and bounds on the initialization error of the backward process when using a Gaussian mixture model with explicitly derived means and covariance matrix. Furthermore, we theoretically analyze the optimal finite stopping time that prevents mode merging during the forward process. Experiments on synthetic and real-world biological data demonstrate that GOU achieves superior structural preservation and sample quality across both statistical and biological metrics.

Primary area:
interpretability and explainable AI
Keywords:
Geometry-aware diffusion, anisotropic diffusion, multimodal data

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VFYLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Geometry-Aware Ornstein–Uhlenbeck (GOU) diffusion process for structured and multimodal data. The main idea is to construct a fixed anisotropic drift matrix MM from the eigendecomposition of the empirical data covariance. Low-variance directions are contracted more rapidly, whereas high-variance directions are preserved for longer during the forward process. For reverse-time generation, the paper approximates the finite-time terminal distribution with a Gaussian mixture whose components are centered around transformed structural anchors. The paper derives the evolution of the mean and covariance, analyzes mode separability, and provides a Wasserstein bound on the terminal initialization error. The method is intuitive, interpretable, and analytically tractable. However, I have significant concerns regarding its novelty, the scope of its theoretical guarantees, the lack of sufficiently strong baselines, its unverified scalability to genuinely high-dimensional data, and the computational cost associated with retraining the score model.

Critical strengths
  1. The drift matrix is explicitly derived from the covariance eigensystem, making the role of each principal direction easy to understand.
  2. Since the proposed process remains a linear OU process, its mean, covariance, and reverse-time dynamics admit explicit characterization.
  3. Preserving multimodal and correlation structure is important for scientific data, particularly genomic and single-cell data.
  4. When important modes are separated primarily along high-variance principal directions, the proposed contraction schedule may delay mode mixing and simplify reverse-time learning.
Critical weaknesses
  1. The paper derives the exact evolution of the first two moments, a finite-time mode-separation result, and an initialization-error bound. These results support the intuition behind the method, but they do not establish convergence of the practical generation procedure to the target distribution. With the exact score, exact simulation, and initialization from the true terminal distribution pTp_T, the reverse-time SDE recovers p0p_0 by standard time-reversal theory. However, this correctness property holds for general sufficiently regular forward diffusions and is not specific to the proposed choice of MM. The practical method instead uses:
  • a finite-sample estimate of MM and μ\mu;
  • a learned score network;
  • a GMM approximation p^T\hat p_T based on anchors;
  • a numerical discretization of the reverse SDE. The paper does not provide an end-to-end error bound that combines these sources of error. In particular, the initialization bound alone does not guarantee that the final generated distribution is close to p0p_0. The paper should distinguish more clearly between guarantees inherited from standard diffusion time reversal and guarantees specifically provided by the proposed drift construction.
  1. The current experiments do not convincingly demonstrate scalability to genuinely high-dimensional settings. The method requires estimating and decomposing a d×dd\times d empirical covariance matrix. When d≫nd\gg n, the empirical covariance is rank-deficient and potentially unstable, while a full eigendecomposition and dense representation of MM may incur substantial time and memory costs. This concern is particularly relevant because the paper motivates the method using genomic and biological applications, which commonly involve tens of thousands or more features. The paper should include:
  • experiments on substantially higher-dimensional datasets;
  • runtime and memory scaling with dd and nn;
  • experiments in the d≫nd\gg n regime;
  • sensitivity to covariance-estimation error;
  • comparisons using shrinkage covariance, truncated PCA, low-rank approximations, and randomized SVD;
  • analysis of how covariance regularization affects generation quality. Therefore, my concern is not that the method has been proven incapable of generating high-dimensional data, but rather that its claimed high-dimensional applicability is currently unsupported.
  1. The empirical evaluation relies mainly on an isotropic diffusion baseline. This is not sufficient to establish the benefit of the proposed approach. Comparisons should include methods designed to incorporate anisotropy, covariance information, structured priors, or preconditioning. In particular, I recommend comparisons against:
  • preconditioned diffusion sampling [1];
  • anisotropic or matrix-valued diffusion methods [2];
  • learned noise or covariance schedules [3]
  1. Although MM and μ\mu are computed directly from data statistics and introduce little additional parameter-learning cost, changing the forward process also changes all intermediate marginal distributions ptp_t and their scores. Consequently, a score model trained for a standard isotropic diffusion cannot generally be reused by simply replacing the drift during sampling. Thus, the method is not a plug-and-play sampling-time modification: it requires retraining, or at least substantial fine-tuning, of the score network under the new perturbation kernel. This is an important practical cost, particularly when a strong pretrained diffusion model is already available.

[1] Ma, H., Zhu, X., Feng, J., and Zhang, L. Preconditioned Score-Based Generative Models. International Journal of Computer Vision, 2025. arXiv:2302.06504.

[2] Voleti, V., Pal, C., and Oberman, A. Score-Based Denoising Diffusion with Non-Isotropic Gaussian Noise Models. arXiv:2210.12254, 2022.

[3] Kingma, D. P., Salimans, T., Poole, B., and Ho, J. Variational Diffusion Models. Advances in Neural Information Processing Systems (NeurIPS), 2021. arXiv:2107.00630.

Questions for authors
  1. Is the proposed GOU process mathematically equivalent to applying a global linear transformation or whitening operation, training a standard diffusion model in the transformed coordinates, and mapping samples back? If not, what is the precise difference?

  2. How much of the reported improvement is attributable to the anisotropic drift, and how much is attributable to the anchor-based GMM initialization?

  3. Does every reported GOU result use a score network retrained from scratch under the GOU perturbation kernel? Can a pretrained isotropic score model be reused or efficiently fine-tuned?

  4. What are the runtime and memory costs of estimating, storing, and applying MM for large dd? Is MM represented densely, or is the eigen decomposition used implicitly?

  5. How stable is the method when d≫nd\gg n and the empirical covariance is rank-deficient?

  6. Why should global high-variance directions generally coincide with directions that are important for mode preservation? What happens when the discriminative structure lies in a low-variance subspace?

3 comments

We thank the reviewer for the careful assessment and for recognizing the interpretability and analytical tractability of our approach. We do our best to respond to your questions point by point:

Is the proposed GOU process mathematically equivalent to applying a global linear transformation or whitening operation?
In general, no. GOU assigns different contraction rates to different principal directions, whereas standard isotropic diffusion uses a shared contraction rate. A fixed invertible linear transformation, including whitening, cannot introduce these direction-dependent contraction rates. This selective contraction can help preserve mode separation when it aligns with slowly contracting directions. Thus, GOU changes the diffusion dynamics beyond a change of coordinates.

How much of the reported improvement is attributable to the anisotropic drift, and how much is attributable to the anchor-based GMM initialization? To address the reviewer's question, we have conducted additional empirical studies comparing GOU with alternative approaches, including isotropic diffusion with a mixture-of-Gaussians initialization. Our new findings clearly demonstrate that the performance advantage of GOU stems not merely from the initialization strategy, but fundamentally from respecting the underlying geometry of the data (please see Lemma 1 and Remark 1). We will integrate these results into the revised manuscript.

Does every reported GOU result use a score network retrained from scratch under the GOU perturbation kernel? Yes. All reported GOU results use score networks trained from scratch under the GOU perturbation kernel. Simply replacing the drift during sampling does not generally yield a valid GOU reverse process, because the intermediate marginal distributions and their scores change. Fine-tuning a pretrained model under the GOU objective is a potential alternative.

Costs of estimating, storing, and applying (M) The cost depends on how M is represented. For (d) features, a dense matrix requires O(d2)O(d^2) memory and O(d2)O(d^2) computation per drift evaluation. A full eigendecomposition costs O(d3)O(d^3), but is performed only once before training. For large datasets, (M) can instead be represented using (r≪dr \ll d) leading covariance directions, with a shared contraction rate for the remaining directions. This reduces storage and per-vector drift computation to O(dr)O(dr), while retaining all (d) dimensions. We have now mentioned these points in our revised manuscript; thanks for raising this question.

How stable is the method when d≫nd\gg n: Rank deficiency does not make the GOU drift singular, because our construction does not invert the empirical covariance. Directions with zero empirical variance receive a contraction rate (-b). Thus, the forward dynamics remain stable. However, when d≫nd\gg n, the estimated covariance may poorly represent the population geometry; shrinkage or low-rank covariance regularization can help address this statistical issue. We will make sure to include this discussion in the paper.

Why should global high-variance directions generally coincide with directions that are important for mode preservation : When clusters are far apart along a particular direction, their separation increases the variance in that direction. This motivates GOU’s slower contraction along high-variance directions, which can help keep clusters separated for longer. However, differences between clusters can also lie in low-variance directions, which the global drift may contract too rapidly. A promising direction for the future is to investigate locally adaptive diffusion using local means and covariances to better capture structure that the global covariance may overlook.

Convergence of GOU: Thank you for raising this point. As mentioned, GOU admits a corresponding backward process under standard time-reversal assumptions, providing a principled foundation for structured data generation. Our main contribution is a geometry-aware diffusion framework for structured data generation, using an anisotropic drift designed to mitigate the decay of feature correlations and delay mode merging during the forward process. Our empirical studies on biological data validate the power of this approach for generating highly structured data. The present analysis characterizes its moment dynamics and directional separation properties and provides a terminal initialization-error bound.

As already acknowledged in the manuscript, a comprehensive non-asymptotic analysis of the practical sampler is planned as a separate, in-depth future study. Further theoretical developments and extensions to higher-order structures remain directions for future research. We will make sure to more clearly state these points in the revised version.

To answer the questions raised by the reviewers regarding: Empirical Evaluation; Scalability to Genuinely High-dimensional; How much of the reported improvement is attributable to the anisotropic drift, and how much is attributable to the anchor-based GMM initialization? we have conducted complementary simulations in our revised version (Please see Appendix; Table 4 and Table 5):

We compare seven approaches: GOU, isotropic diffusion with Gaussian initialization, isotropic diffusion with Gaussian-mixture initialization, truncated-PCA diffusion, and tabular adaptations of PDS (Ma et al., 2025), NI-VP (Voletiet al., 2022), and VDM (Kingma et al., 2021). All methods use the same feature panel, data splits, and training and sampling budgets. Generation quality is evaluated using seven distributional and structural metrics. Results for Paul15 on 500 and 4000 features (high-dimensional setting as n=2700< <d=4000) are represented in Table 4 and Table 5 in the Appendix. GOU well outperforms other methods on most metrics and performs comparably to the strongest baselines on the rest. Our comparison shows that, although method rankings vary across metrics, GOU performs consistently well overall. Furthermore, comparison with isotropic diffusion using Gaussian-mixture initialization suggests that GOU’s advantages extend beyond the initialization strategy. In particular, GOU outperforms the isotropic GMM baseline especially on structural metrics, as it was expected. Similar observations are obtained for other datasets, which will be included in the final version of the paper. We sincerely thank the reviewer for their input.

Drift matrix construction on GOU performance: We have now also added a new section to our Appendix reporting our observations on the computation of matrix M as below:

Impact of drift matrix construction on GOU performance To examine how constructing the drift matrix M affects GOU, we compare full dense covariance eigendecomposition, full-spectrum dual decomposition, and randomized approximations with ranks (256,512)(256,512) and 10241024 on a high-dimensional Paul15 gene panel with 2,700 cells and 4,000 genes; clearly a high-dimensional setup. Each configuration trains a separate score model while keeping preprocessing, drift mapping, architecture, anchor centers and weights, and training and sampling budgets fixed. We assess generation quality and paired changes relative to full dense GOU, alongside computation time and memory usage. Our empirical results showed that randomized approximations with a rank of 512 achieve the best average performance. While providing a marginal change in accuracy, it substantially reduces the storage and computational costs of matrix M (see also lines 212-215 of the paper for a quick discussion).

We hope that our responses have addressed the reviewer's questions and concerns. Please let us know if any points require further clarification.

We have also conducted a quick simulation to compare runtimes in different methods:

Comparing Runtimes

We compare methods using mean total computation time and generation time per 1,000 samples across random seeds. Total time includes method-specific setup, training, initialization, sampling, and evaluation, whereas generation time includes initialization and sampling. For biological data, GOU requires 380 s in total and 55 s per 1,000 generated samples. The corresponding times are 390 s and 58 s for NI-VP, 500 s and 50 s for VDM, 350 s and 50 s for isotropic diffusion with Gaussian initialization, and 50 s and 15 s for truncated PCA. These results show that GOU achieves competitive runtime relative to several baselines, with lower total computation time than NI-VP and VDM, although truncated PCA remains substantially faster but much weaker in performance overall.

Reviewer_CqSFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Geometry-Aware Ornstein–Uhlenbeck (GOU) diffusion, which incorporates the global covariance structure of the data into a fixed anisotropic drift matrix. The forward process follows dXt=M(Xt−μ) dt+β dWtdX_t=M(X_t-\mu)\,dt+\sqrt{\beta}\,dW_t, with contraction rates chosen to retain high-variance directions longer than low-variance directions. For reverse-time generation, the method uses a Gaussian mixture initialization constructed from data anchors at a finite terminal time. The paper derives the evolution of means and covariances and presents theoretical claims concerning initialization error and the preservation of mode separation. Empirical evaluations compare GOU with an isotropic-drift baseline on synthetic Gaussian mixtures, simulated genotype data, and the Paul15 and PBMC single-cell datasets. The reported results show improvements in several distributional and structural metrics, although the gains are not uniform across all datasets and metrics.

Critical strengths
  1. An interpretable and analytically tractable way to incorporate data covariance into diffusion. The covariance-based drift design in Section 3.1 provides explicit control over contraction along principal directions while retaining closed-form expressions for the forward mean and covariance (Theorems 2–3). This is a substantive strength because it offers a concrete, mathematically transparent mechanism for adapting diffusion dynamics to anisotropic data, making the proposed design straightforward to analyze.

  2. Promising empirical improvements in both distributional fidelity and biological structure. On Paul15, Table 1 reports improvements over the isotropic baseline in sliced Wasserstein distance (0.093 vs. 0.437), co-expression Spearman correlation (0.779 vs. 0.454), and cell-state proportion JSD (0.006 vs. 0.236). The agreement across these complementary metrics is meaningful: it suggests that the proposed combination of structured drift and initialization may improve both sample fidelity and the preservation of scientifically relevant dependencies. This empirical promise is a principal consideration in favor of the paper.

Critical weaknesses
  1. The initialization error bound does not apply to the distribution defined in the paper. Equation (12) uses ΣT\Sigma_T as the covariance of each Gaussian component, and Section 5.2 explicitly states that it is computed using Eq. (8). However, Eq. (8) gives the unconditional covariance ΣT=eMTΣ0eM⊤T+QT\Sigma_T=e^{MT}\Sigma_0e^{M^\top T}+Q_T, where QT=β∫0TeMseM⊤s dsQ_T=\beta\int_0^T e^{Ms}e^{M^\top s}\,ds is the conditional noise covariance. The synchronous coupling in Appendix B.6, specifically Eq. (17), generates components with covariance QTQ_T, not ΣT\Sigma_T. Its marginal therefore differs from the initialization distribution in Eq. (12).

This mismatch yields a direct counterexample to Lemma 2 as stated. Let p0=12δ−1+12δ+1p_0=\frac12\delta_{-1}+\frac12\delta_{+1}, use each point as its own anchor, and set M=−0.4M=-0.4, β=0.8\beta=0.8, and T=1T=1. The bound's right-hand side is zero because X^0=X0\hat X_0=X_0, but the true terminal variance is 11, whereas Eq. (12) produces variance 1+e−0.81+e^{-0.8}. The distributions cannot coincide. This invalidates a central theoretical guarantee for the stated initialization scheme. Replacing ΣT\Sigma_T with QTQ_T may repair the coupling argument, but the paper also needs to clarify which covariance was used in the experiments.

  1. Theorem 4's stopping-time condition does not guarantee the stated separation criterion. Combining Theorems 1 and 3 with Eq. (14), and writing ai=∣ci∣>0a_i=|c_i|>0 and si(0)=ui⊤Σ0uis_i(0)=u_i^\top\Sigma_0u_i, gives the exact expression Ri2(t)=αi2si(0)+β2ai(e2ait−1).R_i^2(t)=\frac{\alpha_i^2}{s_i(0)+\frac{\beta}{2a_i}\left(e^{2a_it}-1\right)}. Appendix B.7 drops the initial-variance contribution and then uses a small-aita_it expansion, without establishing conditions that justify both approximations or converting them into a rigorous bound.

For example, take p0=12N(−1,0.01)+12N(1,0.01)p_0=\frac12\mathcal N(-1,0.01)+\frac12\mathcal N(1,0.01), a=10−4a=10^{-4}, β=0.01\beta=0.01, and t=100t=100. This drift is permitted by Eq. (2) with b=1b=1 and γ=10−4\gamma=10^{-4}. Here α=2\alpha=2 and s(0)=1.01s(0)=1.01. The stated condition holds because 100<α2/(2β)=200100<\alpha^2/(2\beta)=200, but the exact expression gives R2(100)≈1.9801<2R^2(100)\approx1.9801<2, contradicting the claimed separation criterion. This occurs even though 2at=0.022at=0.02. Consequently, the current theorem does not substantiate the paper's central claim that its finite stopping-time rule preserves mode separation; the assumptions and time bound require substantive revision.

Questions for authors
  1. Clarification of the implemented initialization and the scope of Lemma 2. In the experiments, was each Gaussian component initialized with the unconditional covariance ΣT\Sigma_T from Eq. (8), or with the conditional noise covariance QT=β∫0TeMseM⊤s dsQ_T=\beta\int_0^T e^{Ms}e^{M^\top s}\,ds? Could you reconcile Eq. (12) with the coupling in Appendix B.6 and address the zero-anchor-error counterexample described above? In particular, does resolving this discrepancy require only correcting the manuscript, or also modifying the implementation and updating the reported results? This distinction would materially affect my assessment of both the theoretical guarantee and the empirical evidence.

  2. Validity and practical use of Theorem 4. Could you address the counterexample above and state a corrected separation condition that retains the initial directional variance si(0)s_i(0)? If the intended result is instead an approximation in a restricted regime, what explicit assumptions and error control justify using it as a sufficient condition for Ri2(t)>2R_i^2(t)>2? Please also clarify the terminal time TT used in the experiments and whether it was selected using this theorem. This would establish which mode-preservation claims remain supported and whether the theoretical issue affects the experimental procedure.

  3. Evidence isolating the contribution of anisotropic drift. What initialization distribution was used for the isotropic baseline? Was it given the same anchors and mixture weights, with component means and conditional noise covariances computed under its own forward dynamics? If such a comparison was performed, could you report whether GOU retains an advantage under comparable training and sampling budgets, specifying the noise levels and terminal times? If it was not performed, what evidence distinguishes the benefit of anisotropic drift from that of the anchor-based mixture initialization? This distinction is important for assessing the contribution of the proposed drift design.

6 comments

We sincerely thank the reviewer for their careful reading of our manuscript and for providing insightful feedback. Below, we address each of the questions raised point by point:

Initialization: We thank the reviewer for raising this point; the reviewer is entirely correct. Equation 12 should indeed have QTQ_T as the covariance of each Gaussian, as used in Lemma 2. We apologize for this typo. We confirm that our implementation already correctly uses QTQ_T (empirical observations remain valid), and this typo in Equation 12 will be corrected in the revised manuscript.

Theorem 4: We thank the reviewer for this careful observation. In the original manuscript, we used a simplified expression (employing ≈\approx notation) to intuitively illustrate the role of weak contraction through the separation ratio. However, we acknowledge that this approximation was insufficiently clarified and led to inconsistencies with the provided example. To resolve this, we have replaced the simplified argument with an exact characterization that retains the initial variance and requires no asymptotic approximations. As the reviewer correctly notes, the exact stopping time can be reached by t∗i=log⁡(1+(ai(αi2−2vi)/β))/2ai{t*}_{i}=\log(1+(a_i(\alpha_i^2-2v_i)/\beta))/2a_i conditioned on initial separation at time zero αi2>2vi\alpha_i^2>2v_i. We also have a plan to include simulations employing t∗t* for terminal time to the manuscript for illustration by the end of the rebuttal period, as in the current simulations t∗t* was not used in our simulations; so, our empirical results remain valid.

Isolating the contribution of anisotropic drift: We thank the reviewer for this insightful question. We confirm that our current simulations employ Gaussian initialization for isotropic diffusion, consistent with the original setting. To address the reviewer's concern, we have conducted additional empirical studies comparing GOU with alternative approaches, including isotropic diffusion with a mixture-of-Gaussians initialization. Our new findings clearly demonstrate that the performance advantage of GOU stems not merely from the initialization strategy, but fundamentally from respecting the underlying geometry of the data (keeping the covariance information; please see Lemma 1 and Remark 1). We will integrate these results into the revised manuscript.

We hope that our responses have addressed the reviewer's questions and concerns. Please let us know if any points require further clarification.

Following up on the raised question regarding isolating the contribution of anisotropic drift, we have now conducted complementary simulations in the Appendix of our revised version. A quick abstract here:

Empirical Comparison Following the reviewers' comments (reviewer kPWN and VFYL), we have now conducted complementary simulations: We compare seven approaches: GOU, isotropic diffusion with Gaussian initialization, isotropic diffusion with Gaussian-mixture initialization, truncated-PCA diffusion, and tabular adaptations of PDS (Ma et al., 2025), NI-VP (Voletiet al., 2022), and VDM (Kingma et al., 2021). All methods use the same feature panel, data splits, and training and sampling budgets. Generation quality is evaluated using seven distributional and structural metrics. Results for Paul15 on 500 and 4000 features are represented in Table 4 and Table 5 in the Appendix. GOU well outperforms other methods on most metrics and performs comparably to the strongest baselines on the rest. Our comparison shows that, although method rankings vary across metrics, GOU performs consistently well overall. Furthermore, comparison with isotropic diffusion using Gaussian-mixture initialization suggests that GOU’s advantages extend beyond the initialization strategy. In particular, GOU outperforms the isotropic GMM baseline especially on structural metrics, as it was expected. Similar observations are obtained for other datasets, which will be included in the final version of the paper. We sincerely thank the reviewer for their input.

We have also included a new empirical section (in the Appendix) to study how applying Theorem 4 can improve our results:

Effect of theorem-based stopping on GOU We investigate the stopping-time criterion in Theorem 4 using two-dimensional Gaussian-mixture data. We compare fixed-horizon and theorem-based GOU with six baseline methods, keeping the drift matrix, training updates, and ... identical between the two GOU settings. We evaluate generation quality and computational cost. As shown in Table 6, the selected stopping time improves or preserves generation quality while reducing the diffusion horizon from T = 1.0 to approximately T = 0.578.

We hope that our responses have answered the reviewers' questions.

Reviewer_kPWNOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the problem that gaussian forward processes do not preserve the correlation structure of data which is crucial for modeling biological sequence. They address this by introducing Ornstein-Uhlenbeck diffusion, and show theoretically how it solves the problem, and validate this with a handful of empirical results.

Critical strengths

Novel approach that solves a clear problem. Well written. Well developed theory.

Critical weaknesses

The major weakness of this paper is that they do not provide empirical comparison between their method and other methods that also aim to account for tight correlation in data. They make several claims in the introduction about the weakness of these other methods, but we have no way of verifying that those claims are true without direct comparison on the same data.

Questions for authors

What is the stability of this type of diffusion like? One of the critiques of diffusion on manifold is that it introduces substantial analytical and computational complexity. What is the computational difficulty of this approach and how does it compare?

Suggestions:

Nit: “and let define U ⊤∆0 =: α” line 242. “Let” appears to be redundant.

The motivating target data for this new method approach is discrete data. This is a method for diffusion in continuous Euclidean space. Of course it is a common approach to do diffusion in Euclidian space with discrete data, but you might want to include the citation for the theoretical justification of this, which I believe was done here: https://proceedings.iclr.cc/paper_files/paper/2026/hash/45d5823950cab9a88e10c6e8dbff351c-Abstract-Conference.html

3 comments

We sincerely thank the reviewer for the positive evaluation and for raising insightful comments. We do our best to answer the raised questions point by point:

Stability and computational complexity: GOU uses a fixed symmetric negative-definite drift, giving a contractive forward process and explicit Gaussian transition kernels without any manifold projections. For nn samples in dd dimensions, dense covariance preprocessing costs O(nd2+d3)O(nd^2+d^3) once. Operator storage and application cost O(d2)O(d^2), reducible to O(dr)O(dr) using r≪dr\ll d leading directions and an isotropic complement. This provides controllable geometric overhead, although reverse-sampling stability still depends on score accuracy and discretization, the same as the isotropic diffusion.

Empirical Comparison Following the reviewers' comments (reviewer kPWN and VFYL), we have now conducted complementary simulations: We compare seven approaches: GOU, isotropic diffusion with Gaussian initialization, isotropic diffusion with Gaussian-mixture initialization, truncated-PCA diffusion, and tabular adaptations of PDS (Ma et al., 2025), NI-VP (Voletiet al., 2022), and VDM (Kingma et al., 2021). All methods use the same feature panel, data splits, and training and sampling budgets. Generation quality is evaluated using seven distributional and structural metrics. Results for Paul15 on 500 and 4000 features are represented in Table 4 and Table 5 in the Appendix. GOU well outperforms other methods on most metrics and performs comparably to the strongest baselines on the rest. Our comparison shows that, although method rankings vary across metrics, GOU performs consistently well overall. Furthermore, comparison with isotropic diffusion using Gaussian-mixture initialization suggests that GOU’s advantages extend beyond the initialization strategy. In particular, GOU outperforms the isotropic GMM baseline especially on structural metrics, as it was expected. Similar observations are obtained for other datasets, which will be included in the final version of the paper. We sincerely thank the reviewer for their input.

We also thank the reviewer for raising writing comments as well as providing us with a valuable reference. We will make sure to implement both in our revised version.

We hope that our responses have addressed the reviewer's questions and concerns. Please let us know if any points require further clarification.

Reviewer_kPWNOpenReview ↗

Major concerns addressed: score increase

Dear Authors, thank you for the additional comparisons. Previously I identified lack of comparisons as the paper's major weakness. Since this has now been resolved, I will increase my score.

We are very pleased that our new experiments addressed the reviewer's questions, and we sincerely thank the reviewer for raising their score and for the valuable discussion.

Comments on the paper

1 comment

We sincerely thank all reviewers for their positive evaluation and constructive feedback. Our revised manuscript includes the following main changes:

Expanded comparisons—Appendix Tables 4 and 5. We compare GOU with isotropic diffusion using Gaussian or GMM initialization, truncated-PCA diffusion, and tabular adaptations of PDS (Ma et al.), NI-VP (Voleti et al.), and VDM (Kingma et al.) across seven statistical and biological metrics. GOU performs consistently well across metrics (over low- and high-dimensional settings with n=2700<< d=4000); its structural improvements over isotropic GMM suggest benefits beyond initialization alone.

Drift construction and computational efficiency New Appendix experiments compare dense, dual, and randomized constructions of (M). Rank-512 randomized approximation provides strong generation quality with reduced storage and computation. Runtime comparisons show that GOU is competitive with several baselines.

Theorem-based stopping—Appendix Table 6. Experiments on 2D Gaussian mixtures show that the selected stopping time preserves or improves generation quality relative to fixed-horizon GOU while reducing the diffusion horizon from (T=1.0) to approximately (T=0.578).

Theoretical clarifications We replaced Theorem 4’s approximate argument with an exact condition retaining the initial variance. We also clarified the scope of our guarantees and the need for score training under the GOU perturbation kernel.

We hope that our responses have addressed the reviewer's questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.