acceptodds
Under review as a conference paper at ICLR 2027

Semantic Score Alignment for Graph Diffusion Models

Abstract

Graph diffusion models have achieved promising performance in graph generation by learning reverse-time denoising processes. Existing approaches typically optimize denoising objectives on noisy node features and edge structures, where the predicted clean graphs implicitly characterize the underlying data distribution. However, conventional denoising supervision mainly focuses on individual reconstruction accuracy and does not explicitly regularize the distributional consistency between real graphs and model-predicted graphs. In this work, we propose semantic score alignment, a distribution-aware regularization framework for graph diffusion models. Motivated by the connection between denoising prediction and data distribution estimation, we introduce an additional score alignment objective to enforce consistency between real graph distributions and predicted graph distributions. Specifically, we first learn a graph-level semantic representation space that captures global structural and attribute information from graphs. The real graphs and predicted graphs are then mapped into this semantic representation space, where their density-induced score functions are estimated and aligned through Fisher divergence. This semantic score alignment provides global semantic guidance for diffusion models beyond conventional reconstruction supervision. Extensive experiments on benchmark graph generation tasks demonstrate that our proposed semantic score alignment consistently improves generation quality across diverse datasets, showing the effectiveness of distribution-level regularization for diffusion-based graph generative modeling.

Primary area:
generative models
Keywords:
Graph Diffusion, Semantic score alignment, Distribution-aware regularization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hmjuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the problem of discrete graph generation. In this work, the authors argue that the existing reconstruction loss for such models has a flaw. That is, while it promotes proper graph reconstruction for a specific sample, it doesn't necessarily learn the data distribution/manifold well. To enforce this, the authors propose a simple regularization loss that first maps each original and reconstructed graph to a latent space. The distribution is then approximated via a KDE, where the regularization loss then encourages alignment between the original and reconstructed graph spaces. They test their method on multiple datasets, showing encouraging results.

Critical strengths
  1. The paper is well written with a good and clear motivation.
  2. The proposed method is very simple and lightweight which is highly useful in practice.
Critical weaknesses
  1. I find the design of the experimental results to be a little strange. The SSA loss is model-agnostic. As such, it makes sense to explicitly see if it can improve the results of various backbone models. The authors briefly test this in Table 1 showing small improvements. However, in the rest of the results, they seem to solely adopt the GRuM backbone (please correct me if I'm wrong). I find this odd because the current results seem to mainly show that it can improve results for GRuM, which while nice, does seem to lessen the impact. Because right now, my main focus is on Table 1 which seems to indicate a very minor boost in performance when including SSA across models.

  2. The authors argue that for existing approaches may not properly learn the data distribution via just a reconstruction loss (lines 54-57). However, the authors seem to base their argument on a regression loss (Section 3.1). However, many prominent methods used a cross-entropy loss (e.g., DiGress, DeFoG [1], Cometh [2], and Disco [3]). As such, it's unclear if this criticism still holds. Could this be why the performance improvement in Table 1 when applied to DiGress is small?

  3. Furthermore, the authors main argument hinges on the fact that MSE loss "does not explicitly constrain whether the collection of predicted samples follows the underlying data manifold" (lines 174-175). However, how true is this in practice? My point being, the authors don't show any empirical evidence that this indeed does happen. They just argue that it could happen. I'd recommend they show evidence that their observations are borne out in reality.

  4. The authors don't report the variances for any of the runs. In fact, it seems that they only fit on one seed for each run. The authors seem to justify this in the appendix with results on different seeds for QM9 and Zinc, however, they both exhibit fairly low variances compared to other datasets (see [1]). As such, I do highly recommend including them because for some datasets like Planar and SBM, it's very unclear to me how statistically significant the results are (the same goes for Table 4). At a glance, it does seem that some may not be.

  5. Some baseline methods are missing including [2, 3] to name a few. I'd recommend including them for a fair and accurate comparison.

  6. The results in Table 4 are missing many strong baselines, e.g., [1, 2, 3], making it hard to compare performance.

[1] "DeFoG: Discrete flow matching for graph generation". ICML 2025.
[2] "Cometh: A continuous-time discrete-state graph diffusion model". TMLR 2025.
[3] Discrete-state Continuous-time Diffusion for Graph Generation". NeurIPS 2024.

Questions for authors
  1. This is a big ask, but detailing results of Model+SSA across all datasets for multiple methods. E.g., DiGress, GruM, DeFoG. Because to me, this is the real test of SSA, because you want it to be something which people will apply to whatever graph generation model they are using. [Weakness 1]
  2. Does the loss critique extend to CE loss? [Weakness 2]
  3. Empirical evidence for distribution misalignment? [Weakness 3]
  4. Variance on multiple datasets? [Weakness 4]
Reviewer_gkC5OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Semantic Score Alignment (SSA), a plug-in regularizer for graph diffusion models that aims to add distribution-level supervision beyond point-wise denoising. A jointly trained GCN encoder maps real and denoised graphs into a shared latent space; a KNN-restricted mini-batch KDE induces an empirical score field (direction proportional to mu(z) - z); and the SSA loss aligns the score residuals of real and predicted latents with stop-gradients on the local centers. The total objective L = L_mse + lambda * L_ssa is optimized jointly over the denoiser and the encoder. Experiments integrate SSA into GDSS, DiGress, and GruM on QM9, evaluate on QM9/ZINC250k and six general-graph datasets, and include ablations on KDE locality and score anchoring plus a five-seed study.

Critical strengths
  1. The motivation is legitimate and clearly articulated: point-wise denoising supervision need not align distributions, and the de Bruijn / Fisher-divergence viewpoint is the right lens for this gap.

  2. The method is simple, architecture-agnostic, and reproducible in outline (explicit loss, algorithm box, per-dataset configuration table).

  3. Two ablations are genuinely diagnostic: local KNN-KDE beats global KDE, and real-anchored score fields beat model-anchored ones.

Critical weaknesses
  1. The training objective is degenerate with respect to the jointly trained encoder. The reconstruction loss does not involve the encoder phi, so phi is trained only by the self-referential SSA loss; under either reading of Eq. (4), the objective admits trivial minimizers (a collapsed encoder makes both residuals vanish; under the printed negative sign, the real-latent term is unbounded below). No anti-collapse mechanism, boundedness argument, or latent-geometry diagnostic is provided. This is consequential because the entire claim of "semantic" supervision rests on the latent space being meaningful; if the space is shaped by the alignment objective itself, the supervision is circular, and the method's mechanism is unexplained.

  2. The empirical comparisons are unreliable at the source level. Two consecutive baseline paragraphs name different sets as the authors' own reproductions ("GruM and DeFoG" vs "GDSS and GruM"); the DiGress QM9 row (FCD 0.3000, NSPDK 0.0006, Scaf. 0.9267) matches no table in the original DiGress paper and conflicts with values reported elsewhere; and the main tables are single-seed while the paper's own five-seed appendix shows variation of the same order as the headline margins (QM9 FCD improvement 0.1238 -> 0.1090 vs a seed range of roughly 0.1087-0.1192). This is consequential because every headline improvement depends on these comparisons.

Questions for authors

Clarification questions (answers could change my assessment):

  1. What prevents the encoder phi from degenerating? Please clarify the exact sign structure and stop-gradient placement of Eq. (4), argue the boundedness of the objective with respect to phi, and show latent-geometry diagnostics (effective rank, embedding norms) over training.

  2. Under the identical pipeline, how much of SSA's gain survives with a plain latent-regression loss ||phi(G) - phi(G_hat)||^2, and with a latent-space MMD or Wasserstein regularizer? This is the decisive control for whether the score-field construction itself contributes.

  3. What value of the KDE bandwidth h was used on each dataset, how was it chosen, and how sensitive are the results? Does the implemented loss retain the 1/h^2 factor (which couples h to the effective lambda)?

  4. Please provide per-row provenance for every baseline entry in Tables 1-4 (original paper with table number / third-party source / your own reproduction with protocol), and reconcile the DiGress QM9 row with the fact that the original DiGress paper reports validity/uniqueness/NLL on QM9 but not FCD/NSPDK/Scaffold.

  5. Given the seed spread in Tables 8-9, which improvements remain statistically significant, and which seed produced each headline number in the main tables?

Additional concerns (for the record):

  1. The abstract states scores are "aligned through Fisher divergence" and the introduction says representations "are extracted through Fisher divergence," but no Fisher divergence is computed; the implemented loss is paired pointwise matching of unnormalized KDE score residuals. Please correct the description or provide the computation.

  2. On Planar, SSA's Orbit MMD (0.0040) is roughly 4x worse than DiGress (0.0009) and 7x worse than DeFoG (0.0006). Above the training-set reference (0.0005), yet the results text never mentions Orbit while claiming consistent improvement. Please explain this pattern.

  3. Table 1 shows GruM's QM9 validity unchanged (99.79 -> 99.79), contrary to the claim that SSA "consistently improves all three backbones."

Suggestions for future work (not part of my decision): a frozen or EMA encoder variant and an explicit anti-collapse regularizer would clarify whether joint encoder training helps; reporting uniqueness and novelty alongside scaffold similarity on molecular benchmarks would strengthen the evaluation.

Reviewer_pWnTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Semantic Score Alignment (SSA), which regularizes graph diffusion models by aligning KDE-based scores of real and denoised graph embeddings. Experiments report improved molecular and general graph generation performance.

Critical strengths

(1) Broad applicability. The evaluation covers two molecular datasets and six general graph datasets, with three diffusion backbones tested on QM9, supporting the method’s generality. (2) Clear improvements over the backbone. Compared with GruM, SSA improves FCD by approximately 12% on QM9 and 31% on ZINC250k (Table 2). (3) Validation of key design choices. The paper compares local versus global KDE and shared versus separate KDE, and provides hyperparameter sensitivity analysis and results across five random seeds (Tables 5–6, Appendix E).

Critical weaknesses

(1) The theoretical interpretation does not match the actual loss. Fisher divergence compares the scores of two distributions at the same location, whereas Eq. (4) compares the score of a single real-data KDE at the real and predicted embeddings. Therefore, the current derivation does not sufficiently support the claimed distribution alignment; additional justification or a more limited theoretical claim is needed. (2) The jointly trained encoder admits a collapsed solution. If all graphs are mapped to the same vector, the SSA loss becomes zero without necessarily improving generation quality. The paper does not explain how this solution is prevented and lacks embedding-variance measurements or a fixed-encoder comparison. This remains an unresolved risk, rather than evidence that collapse actually occurred in the experiments.

Questions for authors

Eq. (4) compares the score of the same real-data KDE at two different embeddings, whereas Fisher divergence compares two distributions’ scores at the same location. What mathematical relationship connects the proposed loss to Fisher divergence, and under what assumptions does minimizing it imply distribution alignment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.