acceptodds
Under review as a conference paper at ICLR 2027

Multi-Domain Discrete Graph Generation

Abstract

Current graph generative models are overwhelmingly trained on a single input distribution and then sampled to reproduce it. In this work, we propose extending graph generation into multiple structural domains. Under our proposed multi-domain setting, existing single-domain models, even when given a domain label, sample every domain from a single shared internal distribution. In preliminary benchmarking, we demonstrate the existing model paradigm fails, even when state-of-the-art models are trained with multi-domain awareness. To address this, we propose Discrete-interpolative Graph Generation or DiG, a graph diffusion model specially-designed to internalize domain-conditioning along a bridged discrete path. Benchmarking demonstrates that DiG achieves at least sampling accuracy across all tested domains, while current domain-conditioned models suffer mode-collapse. Additionally, DiG has the unique capacity for interpolating between domains and extrapolating to unseen structures. This interpolation mechanism uncovers targeted molecule generation for achieving comparable performance relative to a -times larger models.

Primary area:
generative models
Keywords:
graph generation, dicrete graph diffusion, multi-domain graph generation, de novo molecule generation, automated sampling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_v4nLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper contributes:

  • A cross-domain graph generation benchmark, spanning five graph distributions in two domains, molecular and synthetic graphs. Metrics and splits are taken from the original benchmarks, the contribution is to project the samples onto a shared domain.
  • Baseline results in the proposed benchmark achieved by training a large range of existing graph generation models from scratch on the cross-domain dataset
  • A model that performs well on the cross-domain benchmark empirically
  • An analysis showing that their model can extrapolate from domains
Critical strengths
  • The benchmark measures something interesting: How well do graph generation models work on more general tasks?
  • The model works on the new benchmark
  • Very cool result regarding extrapolation
Critical weaknesses

The comparison against existing baselines in Table 1 is weak. The baseline uplift recipe e.g. does not mask the baseline to the respective domain in intermediate stages, and the compute cutoff is conveniently chosen so that their model just finishes while e.g. SPECTRE is terminated prematurely (implied from Table 6). The paper's main claims are not about a comparison with prior models but it should still be noted that the comparison may not be fair.

Questions for authors
  • How many epochs did each baseline training actually complete before the 10-day cutoff?
  • The vocabulary masking mechanism is applied to both nodes and edges, right? Applying it to nodes should not improve structural if I understand correctly, so the edge-masking should be load-bearing. Would you mind publishing results with only node masking and another with only edge masking?
  • I am curious how this scales in the number of domains covered. How does performance look e.g. when only training on 3 or 4 instead of 5 domains? Or if you add a 6th domain like moses or guacamol? What happens if you only consider molecule or only consider graph domains?
  • If you train your implementation of prior models as well as your own model on only one domain, what are the results? It would be good to compare against the results reported by the original papers?
4 comments
AuthorsOpenReview ↗

Rebuttal Part 1/2

Thank you for your review, we appreciate your evaluation on our extrapolation results and are intrigued by your questions on scaling. We include our rebuttal below and hope it addresses all of your weaknesses and questions. We summarized your questions and weaknesses to mark our separate answers, please inform us if any specific weaknesses or questions require further addressing.

W1A: The baseline uplift recipe e.g. does not mask the baseline to the respective domain in intermediate stages

W1A. We note that DiG$^2$'s benchmarking pipeline does constrain node- and edge-vocabularies during all training and sampling steps for each baseline. We have updated the manuscript to say this directly.

W1B, Q1: How many epochs did each baseline training actually complete before the 10-day cutoff (SPECTRE is terminated prematurely (implied from Table 6).)?

W1B, Q1. We note that evaluating models at the 10-day window was determined to ensure consistency in evaluation standards. 10-days was also selected as a relative cut-off, given that 5 other models (DeFoG, DiGress, LGGM, GraphBSI, and GraphRNN) reached the 10,000 epoch cut-off for the paper's Table 1 benchmarking. However, the benchmarking study did attempt to train slower models past the 10-day window, as shown below in the table. We also note that SPECTRE reached epoch 1770 by day 10, with signs of mode-collapse where generated samples consist entirely of hub-structures, instead of successfully generating reference data.

models/epochmax epochs by cutoffmax epochs reached
DiG²87.210,00010,000
DeFoG†53.910,00010,000
DiGress†47.710,00010,000
LGGM46.710,00010,000
GraphBSI†46.610,00010,000
GraphRNN†20.010,00010,000
G2PT167.45,1677,199
Pard†167.05,2747,199
HOG-Diff†110.27,8497,849
GDSS†188.84,5816,699
LGD†127.96,7658,799
SPECTRE†489.51,7702,699

To determine if additional training for SPECTRE can recover signal for multi-domain graph generation, we have trained SPECTRE for an additional five-days and evaluated the model at epoch 2,699. As shown below, SPECTRE with 15 total-days of training has not converged towards a signal for generating graphs across 5 domains and still generates samples with hub-structure.

SPECTREPlanarSBMTreeQM9Zinc
paper0.0/ 27660.0 / 3740.0 / 46117.3 / –9.51 / –
epoch 26990.0 / 29610.0/3650.0/35114.6 / -8.91 / -

Q2: results with only node masking and another with only edge masking?

Q2: Thank you for the suggestion, this rebuttal ran a targeted ablation on node- and edge-vocabulary masking for DiG$^2$ across Planar, SBM, and Tree. As shown below, ablating node-masking for Planar and Tree retains accuracy for the two datasets within the estimated ~8% noise limits on performance. The results also indicate your intuition is correct. The removal of edge-masking is load-bearing on SBM and Planar, reducing performance by 12.5% for SBM and 35.0% for Planar, both of which fall past noise-limits on performance from the paper results.

conditionPlanarSBMTree
both (paper)100%85.0%97.5%
w/o node95.0%77.5%95.0%
w/o edge65.0%72.5%90.0%
neither72.5%70.0%92.5%

Q3: Scaling on 3 or 4 domains? add a 6th domain, moses or guacamol?molecule or only consider graph domains?

Q3. We include updated results demonstrating that DiG$^2$ maintains strong performance across 4 and 6 domains relative to the paper's results. We also include results from molecule-only and synthetic-only training to show that DiG$^2$ is not dependent on comparisons between synthetic and molecular datasets to achieve strong results.

ModelPlanarSBMTreeQM9ZincMOSES
DiG^2(4)97.5/2.185/2.095/1.599.29/0.28N/AN/A
DiG^2(5 - paper)100/2.385/1.897.5/1.699.92/0.2995.46/2.7N/A
DiG^2(6)95.0/2.587.5/1.795.0/1.499.46/0.3194.38/2.493.0/2.1
DiG^2(mol)N/AN/AN/A99.38/0.2295.26/2.494.4/1.9
DiG^2(struct)97.5/1.982.5/2.1100/1.5N/AN/AN/A
AuthorsOpenReview ↗

Rebuttal Part 2/2

Q3 continued.

We also include the MMD ratio results from training DiG$^2$ on a combined LGGM+DiG$^2$ benchmarking protocol with 15 total datasets, as compared to LGGM. As shown below, DiG$^2$ exceeds LGGM performance on 14 of the 15 tested datasets and is within 0.04 of LGGM on the Econ dataset, demonstrating the capacity of DiG$^2$ to scale on additional datasets.

ModelFBASNEmailWebRoadPowerBioEconRTColEcoCitationPlanarSBMTreeAverage MMD ratios (↓)
DiG^21.622.081.502.183.701.481.391.152.373.312.862.0792.5%/1.2085%/2.3097.5%/1.922.08
LGGM2.155.012.273.054.933.781.631.1121.9413.584.522.810.0%/1375.0%/11.30.0%/12.9415.23

Q4: single-domain results? compare against original papers?

Q4. We include single-domain results for DiG$^2$, DeFoG, and DiGress from the per-node budget pipeline and the results for each respective model, as derived from their reported paper results and indicated with a '(paper)' label. The 'DiG$^2$ (paper)' results are from the multi-domain setting. As demonstrated below, the single-domain setting for this study's pipeline enables models to converge towards performance comparable to originally reported results. Notably, the DiG$^2$ pipeline even improves the reported accuracy values in the DiGress paper, making DiGress competitive with a single-domain DiG$^2$ and DeFoG.

ModelPlanarSBMTreeQM9Zinc
DiG^297.5/1.880/2.2100/1.999.45/0.2795.80/2.5
DiG^2(paper)100/2.385/1.897.5/1.5699.92/0.2995.46/2.7
DeFoG95.0/1.685/3.195/1.899.23/0.2795.35/2.2
DeFoG(paper)99.5/1.690/4.996.5/1.698.9/0.2696.65/2.1
DiGress85/1.280/2.395/2.099.4/0.394.16/2.5
DiGress(paper)77.5/5.160.0/1.790.0/1.699.0/----/--

Super cool results on the scaling regarding the scaling in the number of domains: It seems like sometimes the results benefit from other datasets? Add some more ablations and nice graphs regarding this to the paper and I will raise to 3! Very cool! Would be good to understand whether one dataset actually helps performance on the other!

AuthorsOpenReview ↗

Response to Reviewer v4nL – new DiG Figure 3

Thank you for the encouraging response, we also believe that the results indicate the cool capabilities of DiG^2!

We conduct an investigation of the internal structural encoding for DiG$^2$, as shown in the updated PDF manuscript's Figure 3. Within the figure, we find that once training is scaled to include the 12 LGGM domains, the separation between internal Planar and SBM representations improves 10.710.7-times, increasing the gap from 0.03 to 0.32. Additionally, to determine if scaling provides a downstream benefit for Planar and SBM, we then collapsed the fused prototype embedding during sampling towards a per-domain mean, inducing a scenario where DiG$^2$ must rely solely on domain labels for internal model separation. We find this reduces SBM accuracy by 4040 points, from 85% to 45%, even after training with the LGGM datasets. Therefore, indicating that DiG$^2$ sampling benefits from it's structural encoding schema for separating internal per-domain signals and maintaining high-quality performance.

We hope this addresses your concerns and appreciate the excited feedback.

Reviewer_TbJwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addressed the problem of graph generative models on multiple domains. The paper proposed discrete-interpolative Graph Generation. The experiments were evaluated on structural domains (PLANAR, SBM, TREE) and molecular domains (QM9, ZINC).

Critical strengths
  1. The presentation is good.
  2. The motivation is clear: The question of generating multiple structurally distinct graph families with one model is clear and potentially useful.
  3. The idea of conditioning on the domain prior is intuitive.
Critical weaknesses
  1. The definition of 'domain' is not clear, since the paper is motivated and based on 'domain-conditioned', the definition of domain should be clearly formulated, and the 'domain prior' should be discussed with more details.
  2. The domain prior conditioned generative model is not new. IMHO, this paper applies this idea to graph generation with tailored technical adaptation. The novelty for method should be reconsidered.
  3. I am confused by the contribution of "develop the first structurally-distinct multi-domain graph generation benchmark". Is this benchmark simply combining five existing datasets? If so, I did not think this forms a new benchmark contribution; otherwise, more details should be given.
  4. To the best of my knowledge, LGGMS used 13 domains of graph; why did the author not follow this setting or expand based on this prior setting?
  5. In my opinion, the pre-trained models across different domains should be compared with specialized models (otherwise, why don't we use specialized models with faster training and smaller data costs). However, the results in the paper did not show better performance compared to DemoDiff and GenMol.
Questions for authors
  1. How do the authors define a “domain” in this work, and how are the domain-specific priors constructed?

  2. What is novel about DiG2 compared with prior methods that use domain-conditioned generation or domain-specific priors?

  3. What makes the five-dataset benchmark a new benchmark contribution, beyond combining existing datasets? What resources or evaluation protocols are provided?

  4. Why did the authors choose five domains rather than building on LGGM’s 13-domain setting? How do the two settings differ?

  5. How does DiG2 compare with models trained specifically for each domain in generation quality, training cost, and data requirements? What advantages does it offer over GenMol and DemoDiff under comparable evaluation settings?

8 comments
AuthorsOpenReview ↗

Rebuttal Part 1/3

Thank you for your review, we appreciate your attention to the empirical motivation and presentation within our work. We include our rebuttal for addressing all of your weaknesses and questions below. We hope our rebuttal addresses all of your concerns regarding our work. Please inform us if any specific weaknesses or questions require further addressing.

W1, Q1: How do the authors define a “domain” in this work, and how are the domain-specific priors constructed?

W1, Q1. We treat domains (dd) as the dataset with it's feature distribution and defined vocabulary subset (i.e. Planar, SBM, Tree have a shared binary edge vocabulary, QM9 and ZINC have distinct molecular vocabularies).

A domain prior is derived from the class marginal of dd's training graph, as assigned by the 1-hot label for a given domain and constrained by a domain's vocabulary, all of which retains evidence for the target dd in the noisy graph.

The submitted manuscript used domains and datasets interchangeably. We have updated the manuscript to be distinct on the separation between domain and dataset, and explicitly define the domain prior. For example, the synthetic domain (Planar, SBM, Tree) and molecular domain (QM(, Zinc) is now updated to explicitly include the vocabulary, label, and all other components of a referenced dataset .

W2, Q2: What is novel about DiG2 compared with prior methods that use domain-conditioned generation or domain-specific priors?

W2, Q2. LGGM is the only other domain-conditioned graph generation model, and it conditions a shared noise process on a label which pools priors during sampling. Our study is the first to demonstrate that DiG$^2$, with per-domain priors keeps each domain identifiable during denoising. Separately, our study is also the first to integrate node-features within multi-domain graph generation, demonstrating an advancement from LGGM, which was centered around pre-training on the structure from the adjacency matrix alone. We have updated the manuscript to emphasize this point on novelty.

Separately, literature indicates interest in domain-conditioned generation for Drug Discovery [1], CV [2, 3, 4] and even Decision-Making [5]. However, this updated literature review indicates that no previous study considers disjoint node- and edge-vocabularies, while handling domain-specific priors across multiple domains. Therefore, this study is unique by requiring a graph generation model to manage multiple structurally-distinct domains at-once, which DiG$^2$ has achieved successfully.

[1] Hu, B., Park, J. H., Chen, H., Cho, Y. R., & Layton, A. (2025). Domain knowledge infused conditional generative models for accelerating drug discovery. arXiv preprint arXiv:2510.09837.

[2] Khan, U., Nawaz, U., Caputo, M., Bilal, M., Qadir, J., & Khan, M. H. (2026). Keep It Frozen: Domain-Routed Conditional Residual Modulation for Multi-Domain Vision Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 21016-21025).

[3] Belal, A., Meethal, A., Romero, F. P., Pedersoli, M., & Granger, E. (2025, February). Attention-based class-conditioned alignment for multi-source domain adaptation of object detectors. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 8566-8575). IEEE.

[4] Wang, R., Zhang, Y., Zheng, C., Li, C. &; Wu, G.. (2025). A Theory for Conditional Generative Modeling on Multiple Data Sources. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:65618-65654 Available from https://proceedings.mlr.press/v267/wang25eu.html.

[5] Ajay, A., Du, Y., Gupta, A., Tenenbaum, J., Jaakkola, T., & Agrawal, P. (2022). Is conditional generative modeling all you need for decision-making?. arXiv preprint arXiv:2211.15657.

AuthorsOpenReview ↗

Rebuttal Part 2/3

W3, Q3: What makes the five-dataset benchmark a new benchmark contribution, beyond combining existing datasets? What resources or evaluation protocols are provided?

W3, Q3. Intuitively, DiG$^2$'s benchmark is a combination of existing datasets. This is based in the reasoning that DiG$^2$ is built to advance single-domain graph generation towards multiple-domains. Our benchmarking protocol represents a contribution over the single-domain setting and LGGM, given that it does three things differently. The first, it considers multiple structurally-distinct domains at-once, meaning that the benchmarking protocol manages disparate structural and vocabulary input from separate domains throughout the entire pipeline. As shown in our papers benchmarking study, this problem remained unresolved, even with LGGM's contribution. Second, the current single-domain and LGGM protocol applies per-graph batching, whereas the DiG$^2$ protocol executes a per-node batching strategy which also mixes training graphs across domains. Therefore, the DiG$^2$ benchmarking protocol ensures consistent memory pressure, even when handling large graphs. The third, each domain has explicit and varied node- and edge-vocabularies. For example, QM9 has C,N,O,F atom-types within it's node-vocabulary, whereas the ZINC vocabulary adds S, Cl, Br, I, F, and P atoms. So, any generative model within DiG$^2$'s benchmarking protocol must manage disjoint vocabularies to succeed. LGGM is centered exclusively around the binary adjacency matrix structure and includes no node- or edge-vocabulary. All available resources and evaluation protocols are included within the paper's anonymous repository link. We have updated the manuscript to better describe these details and be explicit about the benchmarking contribution as a unique graph generation protocol.

W4, Q4: Why did the authors choose five domains rather than building on LGGM’s 13-domain setting? How do the two settings differ? Why did the author not follow this setting or expand based on this prior setting?

DiG$^2$'s five-domain protocol is an extension on state-of-the-art single-domain models. This is intentional for consistency with single-domain literature and motivated by the question, "Can single-domain graph generative models sample multiple-domains at-once?". Therefore, DiG$^2$ targets generation of structurally-distinctive graph samples between domains, instead of LGGM's pre-training on graph substructure. For example, successful multi-domain generation requires Planar samples to retain delaunay triangulation, SBM to retain community structure, and QM9 samples to retain various atom and edge-types. Initial motivation was also based on how this study's literature review found no previous graph generation model demonstrated success while sampling on multiple structurally-distinct domains.

Initial work also avoided the LGGM pre-training protocol, since it is centered around extracting 2/3-hop subgraphs across LGGM's 12 tested domains, as detailed in Section 3.2 and Appendix C.1 of [1]. Even at 3-hops, extracted subgraphs of SBM samples are capped to 71 out of 188 maximum nodes, eliminating connections between SBM communities and making comparison to the SBM reference graphs from single-domain literature out-of-scope for DiG$^2$'s motivation. Additionally, LGGM's protocol is focused exclusively on adjacency-matrix structure, which ignores node features entirely and makes modeling molecule atom-types impossible, unless integrated with this study's node- and edge-vocabulary. Therefore, DiG$^2$'s use of node- and edge-vocabularies is an extension of LGGM's structure-only training. We include these details within the manuscript to better describe the benchmarking contribution.

[1] Wang, Yu, et al. "A large-scale training paradigm for graph generative models." International Conference on Learning Representations. Vol. 2025. 2025.

AuthorsOpenReview ↗

Rebuttal Part 3/3

W4, Q4 continued.

Given that LGGM's benchmarking protocol is structure-only, it integrates cleanly into the DiG$^2$'s synthetic-domain vocabulary for an extended LGGM+DiG$^2$ benchmarking protocol. We test this combined protocol in the table below. The table calculates MMD ratios (closer to 1.0 is better) across every dataset and includes V.U.N scores (%) for Planar, SBM and Tree. As shown below, DiG$^2$ is the best-performing on 14 of the 15 evaluated domains and remains within 0.04 of LGGM on the Econ dataset. Additionally, DiG$^2$'s average MMD ratio is 5-times lower than LGGM's average MMD ratio, further indicating the DiG$^2$'s strength for generating structurally-distinctive samples.

ModelFBASNEmailWebRoadPowerBioEconRTColEcoCitationPlanarSBMTreeAverage MMD ratios (↓)
DiG^21.622.081.502.183.701.481.391.152.373.312.862.0792.5%/1.2085%/2.3097.5%/1.922.08
LGGM2.155.012.273.054.933.781.631.1121.9413.584.522.810.0%/1375.0%/11.30.0%/12.9415.23

W5, Q5A: (Pre-trained should be compared against specialized) How does DiG2 compare with models trained specifically for each domain in generation quality, training cost, and data requirements?

W5, Q5. To demonstrate how DiG$^2$ compares to specialized single-domain models, we include an additional test with DiGress, DeFoG, and DiG$^2$ on the single-domain setting for the Planar, SBM, Tree, QM9, and Zinc datasets. We distinguish new single-domain results for DiG$^2$ with '(single-domain)' and include multi-domain results with '(paper)'. As shown below, DiG$^2$ is competitive with the single-domain variants of DiGress and DeFoG when trained under the single-domain and the multi-domain setting.

ModelPlanarSBMTreeQM9Zinc
DiGress85/1.280/2.395/2.099.4/0.394.16/2.5
DeFoG95.0/1.685/3.195/1.899.23/0.2795.35/2.2
DiG$^2$ (single-domain)97.5/1.880/2.2100/1.999.45/0.2795.80/2.5
DiG$^2$ (paper)100/ 2.385 / 1.897.5 / 1.5699.92 / 0.2995.46 / 2.7

Separately, we also test DiGress, DeFoG, and DiG$^2$ training time until their respective batches saturate 48GB of GPU memory. As shown below, we find that there is negligible overhead induced by DiG$^2$ when given the same data requirements as DiGress and DeFoG.

ModelPlanarSBMTreeQM9Zinc
DiGress45s52s45s42s44s
DeFoG47s53s44s43s44s
DiG$^2$ (single-domain)47s51s43s44s43s

Q5b: What advantages does it offer over GenMol and DemoDiff under comparable evaluation settings?

To demonstrate the advantage DiG$^2$ provides over GenMol and DemoDiff under comparable standards, we mimic our paper's zero-shot α\alpha-extrapolation setting on GuacaMol's 2-27 node noise bands for 2,000 generated samples. To provide stronger endpoint awareness of QM9 and Zinc, GenMol and DemoDiff are integrated with an embedding adapter which is fine-tuned on QM9 and ZINC for 250k steps. As shown below, DiG$^2$'s V.U.N score is 21.8% higher than GenMol and 39.0% higher than DemoDiff. This improvement translates to FCD, where DiG$^2$ is 6.22 and 3.58 FCD points under GenMol and DemoDiff, respectively. Given that DiG$^2$ is not reliant on rejection-sampling, it provides a 1.95x and 200x boost to average sampling time per-valid molecule over GenMol and DemoDiff, demonstrating consistent gains with improved timing under a comparable setting.

MetricDiG$^2$GenMolDemoDiff
Validity94.7%95.4%89.8%
Uniqueness100.0%81.6%63.9%
Novelty99.8%94.2%96.8%
V.U.N.94.6%72.8%55.6%
FCD ↓10.6017.8214.19
avg. s/per valid sample0.400.7680.1
Reviewer_6tvyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DiG2, a multi-domain discrete graph generative framework meant to address an existing problem in graph generation: models are trained individually on one singular domain, meaning that we end up with one set of weights per domain, and when they are trained on multiple domains at once, models are prone to mode collapse. DiG2 addresses this by conditioning on learned domain embeddings and a prototype computed from actual domain examples, which are injected into the model at every layer, and with per-domain priors as opposed to a single shared distribution. The authors also restrict the vocabulary to the graph's domain during training and sampling to prevent probability assignment outside the given domain and use Classifier-Free Guidance to lead the model towards specific domains at sampling time. Their method achieves high accuracy across domains from a single model. Interpolation and extrapolation between domains are shown to affect molecular generation, with drug-likeness and lipophilicity increasing when interpolating from the QM9 conditioning vector to the ZINC one.

Critical strengths

S1. The paper is written quite nicely and is relatively easy to follow. S2. The topic is timely, in particular about extrapolation to novel molecules. S3. The method is well presented, and the figures help in its understanding. S4. Training and sampling procedures are well described for the most part. S5. The interpolation experiments are interesting and may motivate further work in multi-domain graph generation.

Critical weaknesses

While reading the paper is enjoyable, there are several important points that prevent from recommending acceptance for now:

W1. Novelty needs to be measured: The metrics reported for the structural and molecular datasets look good at first, but novelty is never reported for the main generation results, which is crucial, particularly for the structural datasets. The SBM, planar and tree datasets have so few training examples that it may be easy for the model to simply learn to reproduce them, and no metric in the paper, as far as I can see, tells us that the samples generated in table 1 are actually novel. For the molecular datasets, the paragraphs before and after Table 1 tell us that validity is reported but the actual table says "V.U.N", so I am not sure which one it is. Reporting novelty is absolutely paramount here, in particular since DiG2 has domain-specific prior distributions and prototypes, which could make it prone to overfitting.

W2. On structural domain fine-tuning (alg. 4):

  • What is the performance of the model without fine-tuning ? Why is domain-specific fine-tuning necessary ?
  • Is the performance reported for the protected domains is before or after fine-tuning, this should be disclosed. The effect of the fine-tuning is not measured either: \mathcal{L}_{anc} helps prevent drift in other domains, but there is no guarantee behind this. Reporting results before and after fine-tuning on each domain would be important here.
  • From the anchor fine-tuning code, DiG2 is fine-tuned on new SBM and planar graphs (the DEFAULT_N constant in procedures/pools.py is at 8192 new graphs, which is considerably larger than the original training set). Is this also the case for the other models, or are they fine-tuned on the same 128 SBM/planar graphs that they are originally trained on ? If it is the latter case, this would give an unfair advantage to DiG2 which would see newly generated graphs and thus a more diverse overall dataset. Also, if there is actually fine-tuning on additional graphs beyond the original training dataset, novelty should be measured against those graphs as well.

W3. Comparison with other models:

  • Some of the reported numbers are surprisingly low, such as DeFoG reporting 5% accuracy for SBM, in particular considering that all models are fine-tuned for the structural datasets. Did the comparison models also receive Classifier-Free Guidance ? It is written that vocabulary masking and per-domain priors were applied where applicable, but it would be nice to know which model they were applied to specifically. For example, LGGM has similar per-domain priors when using the domain-specific transition matrix, were they used for the LGGM baseline here?
  • Some other models have extremely high ratios and seem to simply not have learned, such as SPECTRE and LGD. Are you confident that the training pipeline for these models is fair ? For example, LGD's autoencoder is trained for 500 epochs, which may not be enough to learn to encode graphs across many domains, in which case it would never be able to correctly generate graphs. An easy test here would be to check the existing trained autoencoder for reconstruction accuracy, for example. Another good test for the pipeline would be to train a few models on a single domain and see if it leads to comparable performance to what is reported in their respective paper, as check to ensure that the difference between DiG2 and other models comes from architecture and training choices, not from issues with the implementation or simple training hyperparameters.

W4. Classifier-Free Guidance helps here, but since DiG2's architecture is specifically made to improve the conditioning on the domain, in particular through the domain-specific prior and prototype, it would be good to show whether CFG is necessary here, since it affects how much the architectural contributions actually matter. Showing results with different values of g(d) (including g(d)=1) would be very helpful.

W5. For the interpolation experiment, while interesting, the current figure show whether it is the individual sampled graphs that are interpolated or whether the set of sampled graphs at a given \alpha is simply made up of \alpha \times n ZINC-like graphs and (1-\alpha) \times n QM9-like graphs. True interpolation should produce individual graphs that are somewhere in between QM9 and ZINC, but since we are on the aggregate here, it may just be that this is an effect of the distribution of each sampled set.

W6. For the extrapolation, the idea is good and it shows clearly that something is happening for \alpha >1, but I think logP values need to also be compared with the ZINC dataset and not just MOSES, since MOSES filters out molecules with logP values above ~3.5, so the logP values at \alpha = 1.5 may be perfectly in-distribution for ZINC, the dataset it was trained on. Claiming "extrapolate onto molecular distributions well outside its original training distribution" (l473-474) is too strong of a claim based on the current results. Also it would be great to have novelty for \alpha > 1 and for the MOSES 20-27 band.

Some minor comments, which can be easily fixed:

  • There are some typos ("it's" -> "its" l132,410,474,484, "lead" -> "leads" l423)
  • g(d) in equation 6 should be defined, as right now it could really be any domain-dependent scalar. It is defined in the appendix, but it does not say how it is selected.
  • Which validity check is used for SBM is not reported.
  • An analysis of which layer's outputs actually matter depending on the domain would be interesting, while merely a weighted sum of them is provided.

Because novelty is missing from the main results and it is hard to know to which extent the Classifier-Free Guidance and fine-tuning are doing the heavy lifting as opposed to the architectural contributions, the paper seems to be below the acceptance threshold. There are also significant concerns regarding the comparison with other models: it is important to clarify if they were fine-tuned on the same, newly generated graphs that DiG2 was, and that their training was set up in a fair way with respect to competitors.

Questions for authors

Q1. Is the molecular metric in Table 1 validity or V.U.N ? Are novelty and uniqueness reported for the structural datasets ? Q2. What is the performance of the model before and after anchored fine-tuning on the protected and target domains ? Why is this necessary ? Q3. Is the reported performance in Table 1 for the protected domains before or after fine-tuning ? What are the results before and after fine-tuning for each domain ? Q4. Were the comparison models also fine-tuned on the newly-generated SBM and planar datasets ? Or only on the original 128 ? Q5. Did these models receive Classifier-Free Guidance ? Q6. Which models received vocabulary masking and per-domain priors? Q7. Do the baseline models trained on your pipeline reach their published performance when trained on a single domain ? Q8. Do individual sampled molecules actually interpolate between QM9 and ZINC, or is each set at a given \alpha a mixture of QM9 and ZINC-like molecules ? Q9. How do the logP values compare with ZINC's logP distribution ?

8 comments
AuthorsOpenReview ↗

Rebuttal Part 1/3

Thank you for your engaging and thorough review, we appreciate the attention to detail, it is valuable for improving our work's quality. We hope our rebuttal addresses all of your weaknesses and questions. The rebuttal marks each question by summarizing your questions/weaknesses. Please inform us if any specific weaknesses or questions require further addressing.

W1/Q1. Synthetic datasets need novelty, molecule datasets need to specficy novelty or V.U.N? Is the molecular metric in Table 1 validity or V.U.N ?

W1, Q1. We include Uniqueness and Novelty scores alongside Validity (molecules) and Accuracy (synthetic) for the 5 datasets tested within Table 1. The synthetic datasets indicate perfect Uniqueness and Novelty, demonstrating that DiG$^2$ has not memorized any input data. The molecular metric in Table 1 is indeed Validity. We apologize for the typo and have updated the manuscript to reflect Validity for molecular metrics. We include detailed results adding Uniqueness and Novelty within the updated appendix. Note, QM9 indicates 50.15% Novelty given that the dataset is designed for reconstructing the training data split.

DiG$^2$PlanarSBMTreeQM9Zinc
Accuracy/Validity100%85%97.5%99.92%95.46%
Uniqueness100%100%100%97.28%99.95%
Novelty100%100%100%50.15%99.95%

Q2, W2. What is the performance of the model without fine-tuning ? Why is domain-specific fine-tuning necessary ? Is the performance reported for the protected domains is before or after fine-tuning? Are models fine-tuned on the newly-generated 8192 graphs?

Q2, Q3, W2, Q4. The performance within the original submission's Table 1 is reported after fine-tuning for every model, whether it was anchored or domain-specific. We applied fine-tuning to boost performance for all benchmarked models. The anchored-KL term was introduced for stability during the fine-tuning process to limit impacts on downstream performance of untargeted domains. The inclusion of anchored fine-tuning was motivated by the observation that fine-tuning can reduce performance on domains not targeted by fine-tuning, as shown by how fine-tuning DiGress and LGGM lowers Planar accuracy by 12.5% and 7.5%, respectively. Separately, fine-tuning with Planar, SBM, and Tree for all models integrates the updated 8192 generated graphs, ensuring that all models within this paper's study receive the same chance to boost performance on the synthetic datasets. We will update the manuscript to clarify these points.

We include results for DiG$^2$, DeFoG, and DiGress 'Before' and 'After' fine-tuning within the tables below. The 'After' columns reflect values from the original submission's Table 1.

DiG$^2$PlanarSBMTreeQM9Zinc
Before100/2.677.5/4.195/1.898.36/0.4191.94/2.9
After100/2.385/1.897.5/1.699.92/0.2995.46/2.7
DeFoGPlanarSBMTreeQM9Zinc
Before77.5/7.612.5/8.457.5/2.788.57/0.8390.41/3.7
After95/4.35.0/6.962.5/2.490.21/0.4992.93/3.4
DiGressPlanarSBMTreeQM9Zinc
Before22.5/12.927.5/4.245/2.3092.79/0.7086.8/5.1
After10.0/7.837.5/3.067.5/1.895.43/0.4187.1/4.7
LGGMPlanarSBMTreeQM9Zinc
Before7.5/20.110.0/9.50.0/9.685.9/0.9277.4/9.07
After0.0/7612.5/9.10.0/8.886.3/0.7585.2/8.55

Q5, W3.1, W4. Comparison with other models: Classifier-Free Guidance on comparison models, on LGGM? Showing results with different values of g(d) (including g(d)=1) would be very helpful.

Q5, Q6, W3.1, W4. All models within the paper's Table 1 benchmarking integrate vocabulary masking. Per-domain priors are only integrated into DiGress and DeFoG, since both models train with the same denoising Graph Transformer backbone as DiG$^2$. Classifier-Free guidance is applied on: DiGress, DeFoG, LGGM, and LGD. Within the original study, we set CFG weight for SBM at ω\omega = 3.0 and all other domains at ω\omega = 2.0 across DiG$^2$, DiGress, DeFoG, LGGM, and LGD. We include a grid search on g(d) for DiG$^2$ below. We have also updated the manuscript with these details directly.

Shown below, we include performance for DiG$^2$ across g(d) = 1 and an unconditional variant, showing CFG is most helpful for SBM performance, boosting accuracy by 40% relative to the unconditional variant. The table also demonstrates that gradually ablating CFG monotonically-decreases performance within this test.

CFGPlanarSBMTreeQM9
Paper g(d)100/2.385.0/1.897.5/1.699.92/0.29
g = 1 (no guidance)97.5/2.482.5/3.592.5/1.899.74/0.31
Unconditional92.5/2.745/10.987.5/2.099.21/0.33
AuthorsOpenReview ↗

Rebuttal Part 2/3

W3.2. Is LGD evaluation fair, trained autoencoder w/ reconstruction accuracy?

W3.2. We pre-trained LGD's autoencoder for 500 epochs since LGD's latent reconstruction accuracy was greater than 95% by epoch 400. LGD achieved 100% node and edge reconstruction by epoch 500, as shown below:

domainnode accuracyreal-edge accuracyall-pair accuracy (incl. no-edge)'trivial' floornodesreal edges
Planar100%100%100%91.3%2,04811,376
SBM100%100%100%91.1%2,08219,460
Tree100%100%100%96.9%2,0484,032
QM9100%100%100%85.7%2,2854,468
Zinc100%100%100%91.0%2,1494,618

Given LGD's results while benchmarking, LGD's performance is evidence that LGD is a powerful latent reconstruction model, far exceeding the trivial floor for predicting latents across each tested domain. However, it currently appears difficult to translate latent reconstruction to latent diffusion for multi-domain graph generation.

Q7. Single-domain on multi-domain protocol?

Q7. In order to verify whether our benchmarking protocol can lead to effective single-domain performance for baseline models, we trained and evaluated DeFoG and DiGress within our current pipeline on one dataset at a time. As shown below, DeFoG is within margins on accuracy and MMD ratio for each domains and DiGress exceeds it's originally reported results, indicating that our pipeline's problem setting is consistent in single-domain generative tasks.

DeFoGPlanarSBMTreeQM9Zinc
single-domain95.0/1.685/3.195/1.899.23/0.2795.35/2.2
paper99.5/1.690/4.996.5/1.698.9/0.2696.65/2.1
DiGressPlanarSBMTreeQM9Zinc
single-domain85/1.280/2.395/2.099.4/0.394.16/2.5
paper77.5/5.160.0/1.790.0/1.699.0/----/--

W5. True interpolation should show graphs somewhere between QM9 and Zinc and Q8. is each set at a given \alpha a mixture of QM9 and ZINC-like molecules ?

W5, Q8. In order to empirically investigate whether the α\alpha-conditioning protocol for DiG$^2$ interpolates between QM9 and Zinc, we consider the delta for Bayesian information criterion (BIC) between and within the mixture of the projected Fisher Direction for our generated QM9 and ZINC endpoints. We choose BIC and the Fisher direction since both produce a clean mean axis score and can show overlap outside interpolated distributions, with a standard deviation for whether the mixture spreads toward distinct endpoints. As shown below, we determine that the mean axis score rises in correlation towards ZINC as the endpoints move from QM9 to ZINC. We also see the observed standard deviation spread stays within tight margins relative to the spread of a gaussian mixture, a full 0.27 tighter at α\alpha = 0.5. This is further supported by how Δ\DeltaBIC indicates clear positive scores across all α\alpha values and achieves +4044+4044 BIC at α\alpha=0.5, indicating that 78.678.6% of generated samples cannot be distinguished to either endpoint.

αmoleculesmean axis scoreobserved spread (sd)gaussian mixture spread (sd)ΔBICverdict
0 (QM9 endpoint)2,5830.000.40——reference
0.505,5960.580.340.61+4044interpolation
0.755,6550.810.320.40+1557interpolation
0.855,6540.890.320.35+559interpolation
0.955,6400.970.310.32+40interpolation
1 (ZINC endpoint)2,8291.000.31——reference

Q9, W6. logP values at \alpha = 1.5 may be perfectly in-distribution for ZINC. novelty for \alpha > 1 and for the MOSES 20-27 band would be helpful.

Q9, W6. We run an α\alpha > 1 extrapolation test on logP values for the ZINC endpoint across the 20-27 band. As shown below, the median logP value for all α\alpha greater than 1.00 contributes to a monotonic increase which reaches a median logP = 4.82 at α\alpha=2.0. α\alpha=2.0-generated molecules also sit at 59.2% above the 95th-percentile of ZINC's logP reference set, all while Novelty remains at 100% across every α\alpha-interpolation value. This is further compounded by the Mahalanobis centroid distance relative to the ZINC endpoint, achieving a direct 2.77 gain over the α\alpha=1.0 distance. All of which indicates that α\alpha-extrapolated molecules DiG$^2$ move beyond ZINC's initial logP distribution.

setnlogP median% above ZINC p95NoveltyMahalanobis centroid vs ZINC
ZINC reference38,3542.92———
α = 1.005,5802.895.4%100%0.90
α = 1.255,4513.2511.2%100%0.81
α = 1.505,3373.6622.8%100%1.34
α = 1.754,8754.2141.2%100%2.30
α = 2.004,0064.8259.2%100%3.67
AuthorsOpenReview ↗

Rebuttal Part 3/3

W7. typos ("it's" -> "its" l132,410,474,484, "lead" -> "leads" l423)

A7. Thank you for the catch, we have updated Lines 132, 410, 474, 484, and 423 within the manuscript with the proper punctuation.

W8. g(d) in equation 6 should be defined, as right now it could really be any domain-dependent scalar. It is defined in the appendix, but it does not say how it is selected.

A8. We have updated the manuscript to include an explicit definition on g(d), as follows for Lines 254-256: "Both predictions are linearly extrapolated with a domain-specific strength, where g(d) ∈R\in \mathbb{R} is a scalar guidance weight read from a per-domain lookup table"

W9. Which validity check is used for SBM is not reported.

A9. We have updated the manuscript to be explicit about how we follow SPECTRE's SBM validity check function, which measures whether generated samples have: 2-5 blocks, block sizes of 20-40, and a Wald test with a p-threshold > 0.9 on reference inter- and intra-community connection probability.

W10. An analysis of which layer's outputs actually matter depending on the domain would be interesting, while merely a weighted sum of them is provided.

A10. We conduct a more-detailed ablation on the per-layer prediction configuration for DiG$^2$. The prediction gate determines which blend of layer 8, 10, and 12 are integrated into the final denoising prediction. We demonstrate results for readout predictions at the three specified layer-depths, the uniform mean of those prediction depths, and the gate below. From the results, we observe that the learned gate allows DiG$^2$ to achieve consistently high-performance for the five tested domains.

readout typePlanarSBMTreeQM9ZINC
gate (ours)100%85%97.5%99.92%95.5%
depth 8 only65%60%95%99.6%95.1%
depth 10 only80%67.5%100%99.7%96.0%
depth 12 only (final layer)82.5%80%72.5%91.7%90.8%
uniform mean of depths80%80%85%98.0%92.8%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.