acceptodds
Under review as a conference paper at ICLR 2027

Manifold Alignment for Graph Diffusion

Abstract

Graph diffusion models are a powerful framework for generating complex structured data. However, existing training objectives such as denoising score matching or squared error minimization operate in a sample-wise manner, which optimize individual samples independently and overlook the underlying manifold geometry of the data distribution. To address this limitation, we propose manifold alignment learning (MAL) for graph diffusion models. MAL aligns the real and generated data distributions by non-parametric kernel density estimation on batch samples, while using a semantically informed embedding space solely to compute kernel similarities. By the MAL, we derive a consistency objective that encourages generated graphs to align with the real data manifold. Consequently, MAL promotes manifold-level distributional alignment at each diffusion step. Extensive experiments on graph benchmarks demonstrate that augmenting baseline graph diffusion models with MAL consistently yields comparable generation quality.

Primary area:
generative models
Keywords:
Manifold Alignment, Graph Generation, Graph Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_CrrQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Manifold Alignment Learning (MAL), an auxiliary regularization objective for graph diffusion models. The authors argue that conventional sample-wise denoising objectives may not adequately preserve the global distributional structure of graph data. To address this, MAL estimates empirical score fields from minibatches of real graphs and denoised predictions using kernel density estimation (KDE), with kernel similarities computed through a pretrained graph encoder. The resulting score-field discrepancy is incorporated into the original diffusion training objective without modifying the backbone architecture or sampling procedure. The method is evaluated on four graph generative backbones (GDSS, GruM, DiGress, and DeFoG) and six molecular and general graph benchmarks. The experiments report improvements in several generation-quality metrics, including molecular validity, FCD, and graph structural statistics. The paper also provides hyperparameter sensitivity analyses and additional multi-seed results on QM9.

Critical strengths
  1. Simple and broadly applicable regularization framework.

    A notable strength is the modularity of the proposed approach. MAL can be integrated into different graph generative frameworks without modifying their underlying architectures or sampling procedures. The experiments across both continuous and discrete graph generation models support its practical applicability. This flexibility makes the approach potentially useful as an auxiliary training objective.

  2. Encouraging empirical results across multiple benchmarks.

    The paper reports consistent improvements on several generation-quality metrics across multiple backbones and datasets. In particular, the improvements in FCD and molecular validity on QM9 and ZINC250k are encouraging. The additional evaluations on general graph benchmarks further demonstrate the potential applicability of the approach beyond molecular generation.

Critical weaknesses
  1. The Gaussian-kernel derivation does not cover the implemented field. Appendix A.4 derives the score identity using Gaussian weights and displacements in the same coordinate space: the raw displacement follows from differentiating the squared Euclidean distance in the kernel exponent. Section 3.3 then substitutes encoder-space weights, while Eq. 9 retains raw-coordinate displacements. Differentiating the encoder-space KDE with respect to the raw input instead introduces the encoder Jacobian and feature-space displacements, which are absent from Eq. 9. The same centroid formula therefore does not generally remain a score after this substitution. A justification is needed for the encoder-based field, since its score interpretation provides the mathematical basis for the paper's distribution-alignment claim.

  2. The link from prediction alignment to correcting reverse sampling needs justification. Section 3.1 attributes sampling distortion to using a single conditional-mean prediction, while Eq. 9 aims to align the marginal distribution of predictions across a batch with the data distribution. A reverse transition depends on how each clean prediction relates to the current noisy state; matching prediction marginals alone does not determine that relationship. Appendix A.3 bounds discrepancy after applying a shared forward-corruption kernel to clean outputs and explicitly notes that it does not directly characterise reverse sampling. It therefore does not establish the claimed correction of the reverse transitions. Figure 2 compares the full regulariser with the baseline, Figure 3 varies hyperparameters, and Appendix C shows final samples and nearest neighbours. These results support practical gains, but do not directly test whether the proposed alignment reduces errors along the actual reverse trajectory, as claimed in Section 4.4. Evidence connecting the regulariser to this specific effect is needed.

Questions for authors
  1. Score construction.

    Which density and coordinate space define the score represented by Eq. 9 with the encoder-based kernel? How does its raw-coordinate weighted displacement follow from the latent-space score or its gradient through the encoder?

  2. Connection to the stated sampling problem.

    What specific approximation or estimation error causes the distortion described in Section 3.1, and what evidence shows that the combined objective reduces it? How does the forward-corruption bound in Appendix A.3 support this correction during actual reverse sampling?

Reviewer_VncTOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Manifold Alignment Learning (MAL), which uses a frozen pretrained graph encoder to compute kernel similarities and constructs a score-field regularization loss from batches of real and predicted graphs. This regularizer is added to existing graph diffusion objectives to improve distributional consistency and structural quality.

Critical strengths

The method is simple and intuitive, and can be integrated into multiple graph generation backbones.

Critical weaknesses
  1. A key argument motivating the method is insufficiently justified. The paper links conditional means lying outside the valid graph manifold to accumulated structural distortion during reverse sampling. However, a conditional mean is a statistical quantity used to construct the score or reverse dynamics and need not itself be a valid discrete graph. The paper does not sufficiently explain when this property of intermediate predictions harms the final generated distribution, or how MAL specifically addresses this problem.

  2. The theoretical connection between the regularizer and manifold alignment remains unclear. Equation (9) more directly matches the differences between samples and their respective weighted semantic centroids. This does not by itself establish equivalence to standard Fisher divergence. How kernel similarities in the semantic representation space correspond to the scores computed from node features and adjacency matrices also requires clarification. The current theory does not sufficiently support the paper’s strong interpretation of manifold alignment or its necessity.

  3. The contributions of the key design choices have not been adequately isolated. The ablation study mainly compares models with and without the complete MAL regularizer. It therefore does not distinguish gains from score-field regularization, pretrained representations, or simpler distribution-level or batch-level regularization. The frozen graph encoder directly determines kernel similarities, yet the paper does not report comparisons using alternative encoders or removing pretraining on the same dataset.

Questions for authors
  1. Could the authors clearly explain the relationship between Equation (9), the claimed Fisher divergence, and manifold alignment?

  2. Could the authors include simple comparisons such as MMD and uniform neighborhood weights under the same encoder and backbone with comparable training and hyperparameter tuning budgets, and add experiments with alternative encoders to distinguish the contributions of the representations, the local matching objective, and the overall regularization?

Reviewer_TwfjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Manifold Alignment Learning, a training-time regularizer for graph diffusion models. It argues that sample-wise denoising collapses the multi-modal posterior into one predicted clean graph, so reverse trajectories can drift off the data manifold. MAL estimates batch-level densities of real and generated graphs with Gaussian kernel density estimation, using a frozen self-supervised graph encoder only to compute kernel similarities, and aligns the resulting empirical score fields through a Fisher-divergence-style loss added to the standard denoising objective. It is integrated into four backbones and evaluated on two molecular and four general graph benchmarks, reporting consistent gains in validity, distributional distances, and structural MMD metrics.

Critical strengths
  1. Clear problem framing: point-estimate denoising can anchor reverse trajectories off the data manifold, motivating batch-level regularization.
  2. Genuinely plug-and-play: one training-only loss adapts to four backbones, covering continuous and discrete diffusion, with unchanged inference.
  3. Broad evaluation: two molecular and four general graph benchmarks, standard metrics, and joint sensitivity analysis over regularization weight and kernel bandwidth.
Critical weaknesses

1.The implemented loss is not the claimed Fisher divergence: scores sit at different points, kernels are latent while score vectors are raw, and stop-gradient reduces it to a batch-dependent regression target. The theory is internally inconsistent: the paper proves standard denoising already equals integrated Fisher divergence, while its contraction bound covers forward noising only, not reverse sampling.

2.Reporting is uneven: single-seed main results, headline FCD below the five-seed mean, non-significant scaffold gains, and Table 1's universal improvement claim fails for DeFoG.

Questions for authors

See Weakness.

Reviewer_JypBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes manifold alignment learning (MAL) for graph diffusion models. MAL aligns the real and generated data distributions by non-parametric kernel density estimation on batch samples, while using a semantically informed embedding space solely to compute kernel similarities.

Critical strengths
  1. This method involves incorporating a training regularization term into existing graph diffusion models, rather than designing an entirely new generative model.
  2. Standard loss functions primarily assess the accuracy of individual image reconstructions. The authors argue that this is insufficient; one should also examine whether the distribution formed by a collection of generated images aligns with the distribution of real images.
Critical weaknesses
  1. In the experiments, the addition of MAL led to improvements in molecular validity, distribution similarity, and general graph-structural metrics for some models; however, not all metrics showed improvement, and there is no guarantee that every generated graph will be valid.
  2. Does MAL, while enhancing overall distributional similarity, preserve structural heterogeneity within groups—specifically minority patterns and localized structural differences?
Questions for authors
  1. When does MAL fail to improve generation quality, and does it guarantee individual graph validity?

  2. Does MAL preserve minority patterns and local structural diversity, or could alignment favor dominant patterns?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.