acceptodds
Under review as a conference paper at ICLR 2027

GRAFT: Relational Node Fields for Latent Graph Generation

Abstract

Graph generation is challenging because structural relations are inherently pairwise, leading many diffusion and flow models to maintain adjacency, edge, or pair-level states throughout generation. Latent graph generation offers a more compact alternative, but existing approaches often lack relational representations rich enough to preserve graph structure, limiting their ability to realize the scalability benefits of node-only transport as graph size grows. We propose GRAFT, a latent graph generation framework that constructs relational node fields and transports them using Flow Matching. GRAFT folds pairwise and global relational information into node representations while keeping the entire generative trajectory node-only, with self-conditioning used to improve node-field transport. GRAFT achieves competitive generation quality across diverse graph domains while substantially improving sampling efficiency. On Protein, GRAFT samples a graph in s with MiB of peak GPU memory under each method's native inference configuration, i.e. to faster and to lower in peak memory than the evaluated methods.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Generation, Latent Diffusion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_cQ2vOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper improves a two-step graph generation process (LG-Flow) where the first step encodes a graph into node-only latent space with an auto-encoder and the second step generates latent space samples, where during inference the second step-generated samples are fed into the decoder part of the auto-encoder. The specific improvement is their auto-encoder mechanism captures non-adjacent dependencies better. Empirically, they show competitive performance at substantial reductions in compute requirements at inference. They further make theoretical statements on error amplification.

Critical strengths
  • Clearly articulated
  • Strong empirical results
Critical weaknesses
  • Missing SOTA baseline: GraphBSI [1] outperforms GRAFT in several key metrics e.g. on GuacaMol and MOSES.
  • The novelty is narrow: An optimization to the autoencoder and a bit of theory.

[1] Petersen, Ole, et al. "Discrete Bayesian Sample Inference for Graph Generation." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors

Would you mind ablating the performance like in Table 5 but using LG-Flow's encoder with your decoder and your encoder with LG-Flow's decoder? This would isolate performance gains.

4 comments
AuthorsOpenReview ↗

Response to Reviewer cQ2v (Part 1): Comparison with GraphBSI

Thank you for your careful review and for recognizing the clarity of the presentation and the empirical results. We appreciate your suggestions for adding a stronger baseline and isolating the contributions of the relational encoder and decoder.

(W1) Comparison with GraphBSI on GuacaMol and MOSES

"Missing SOTA baseline: GraphBSI [1] outperforms GRAFT in several key metrics e.g. on GuacaMol and MOSES."

Thank you for identifying GraphBSI as an important baseline for these molecular benchmarks. We apologize for its omission from our original comparisons. Following this suggestion, we evaluated the official GraphBSI checkpoints on MOSES and GuacaMol to provide a comprehensive comparison of both generation quality and computational scalability.

(1) Generation Quality: Complementary Strengths in Local Validity vs. Global Distributional Fidelity

We evaluated both methods using the same molecular evaluation toolkit, generating 10,000 molecules per configuration with 100 network function evaluations (NFEs). For GraphBSI, we evaluated both Euler–Maruyama (EM) and Ornstein–Uhlenbeck (OU) sampling.

Table R6(a): Molecular generation quality on MOSES (100 NFEs).

MethodValidity ↑VUN ↑FCD ↓
GRAFT94.38%87.17%0.7987
GraphBSI (EM)99.48%94.85%1.0315
GraphBSI (OU)99.66%94.83%1.0751

Table R6(b): Molecular generation quality on GuacaMol (100 NFEs).

MethodValidity ↑VUN ↑FCD Score ↑KL ↑
GRAFT93.51%92.26%84.0497.69
GraphBSI (EM)98.59%98.30%74.0192.27
GraphBSI (OU)98.80%98.44%76.3793.96

Table R6(c): Sampling time per graph and estimated cost of generating 10,000 molecules (100 NFEs; lower is better).

DatasetMethodTime (s/graph) ↓Estimated 10K Time ↓
MOSESGRAFT0.01983 min 17 s
MOSESGraphBSI (EM)0.13014221 min 41 s
GuacaMolGRAFT0.0323705 min 24 s
GuacaMolGraphBSI (EM)0.7897892 h 11 min 38 s

Runtime is measured on the same RTX 4090 GPU with batch size 32, FP32 precision, and 100 network evaluations.

As shown in Tables R6(a) and R6(b), the comparison reveals a clear, fundamental trade-off between local syntactic validity and global distributional fidelity: while GraphBSI achieves higher syntactic Validity and VUN, GRAFT substantially outperforms GraphBSI on the primary distributional metrics—reducing FCD by 22.6%–25.7% on MOSES (0.7987 vs. 1.0315/1.0751) and improving FCD Score by +7.67 to +10.03 points (84.04 vs. 74.01/76.37) and KL Score by +3.73 to +5.42 points (97.69 vs. 92.27/93.96) on GuacaMol (and further reaching 88.16 FCD Score at 500 NFEs in Table R3).

This trade-off stems directly from how each paradigm represents graph structure. GraphBSI iteratively refines categorical distributions over explicit atom and bond types at every step, making it easy to satisfy local valency rules (Validity), whereas matching complex global property distributions (FCD and KL) in high-dimensional discrete space is more challenging. In contrast, GRAFT performs continuous straight-line Flow Matching over compact relational node fields, enabling markedly stronger global distribution alignment (FCD and KL) across chemical and physicochemical properties. Although atom types, formal charges, hydrogen counts, aromaticity, and bond relations are supervised during Stage-I reconstruction, strict valency rules are not explicitly constrained during Stage-II transport or single-pass decoding. Incorporating chemistry-aware valency constraints into relational decoding is a natural extension to further boost validity while retaining GRAFT's strong distributional fidelity and speed.

(2) Computational Efficiency: Practical Benefits for Large-Scale Molecular Sampling

More importantly, comparing runtime scaling from MOSES to the larger molecules in GuacaMol highlights the architectural advantage of GRAFT's O(n)O(n) node-only transport: whereas GRAFT's per-graph sampling time increases by only 1.6× (0.0198s → 0.0324s), GraphBSI's runtime surges by 6.1× (0.1301s → 0.7898s) because it must maintain and update dense O(n2)O(n^2) edge states at every sampling step. This advantage is critical in practice, as standard molecular benchmarks require generating 10,000 or more molecules per evaluation. Extrapolating from the measured batch throughput, generating 10,000 molecules takes 3.3 minutes with GRAFT versus 21.7 minutes with GraphBSI on MOSES, and 5.4 minutes versus 131.6 minutes (over 2.1 hours) on GuacaMol. We have added these comparisons and discussions to the revised manuscript.

[1] Petersen, O., Kollovieh, M., Lienen, M., and Günnemann, S. Discrete Bayesian Sample Inference for Graph Generation. ICLR, 2026.

AuthorsOpenReview ↗

Response to Reviewer cQ2v (Part 2): Contributions and Cross-Ablation

(W2) Contribution beyond an Autoencoder Modification and Theoretical Interpretation

"The novelty is narrow: An optimization to the autoencoder and a bit of theory."

We appreciate the reviewer’s concern and have revised the manuscript to make our conceptual and technical advances over prior node-latent work (such as LG-Flow) more explicit. Rather than an isolated autoencoder tweak plus standalone theory, GRAFT introduces a tightly coupled framework that resolves key limitations of prior node-latent generators on strict structural benchmarks and large graphs.

First, relational field construction resolves a fundamental representational bottleneck in node-only compression. Prior node-latent methods (e.g., LG-Flow) fail to preserve exact multi-hop topology when compressing edges into node vectors, yielding 0.00% exact graph reconstruction on both SBM and Protein. GRAFT overcomes this by coupling multi-level relational encoding with contextual neighbor-aware decoding (see Reviewer QAQr, Q2.2). As demonstrated by the cross-swapping ablation in Q1 (Table R7), neither the encoder nor the decoder alone suffices; their synergy boosts exact reconstruction from 0.00% to 90.63% on SBM and 93.75% on Protein, making high-precision node-only generation viable.

Second, our theoretical analysis identifies and quantifies a critical failure mode in two-stage node-field generation: transport error amplification. Even with a near-perfect Stage-I autoencoder, Stage-II must generate continuous node fields imperfectly via ODE integration. Proposition 1 characterizes how small terminal transport perturbations in the node field can propagate into decoded relational errors through the relational decoder Jacobian, including their dependence on graph size. We further examine this phenomenon empirically using real terminal transport errors on Protein (Fig. 4).

Third, this error-amplification analysis motivates our investigation of Stage-II self-conditioning to stabilize relational trajectories. Because decoded edges are sensitive to endpoint perturbations (Proposition 1), feeding back the estimated clean node field during Flow transport can promote earlier stabilization of the predicted relational structure, as supported by the reduced relational drift in Fig. 5(d). Using the same self-conditioning-trained checkpoint, enabling inference-time feedback reduces the Protein generation Ratio by nearly 3× (from 14.75 to 5.10, Table 6).

Together, these contributions connect relational field construction, error-amplification analysis, and self-conditioned transport. Their scalability is further evaluated on Point Clouds (Reviewer QAQr, W1).

(Q1) Cross-Swapping GRAFT and LG-Flow Encoders and Decoders

"using LG-Flow's encoder with your decoder and your encoder with LG-Flow's decoder?"

"This would isolate performance gains."

Thank you for proposing a direct way to separate the encoder and decoder contributions. To isolate the contributions of the encoder and decoder, we conducted cross-component ablations on SBM and Protein, combining LG-Flow (L) and GRAFT (G) components under the same target training-step budget (98K for SBM and 200K for Protein), using a shared 32-dimensional latent space and reconstruction objective.

Table R7: Stage-I encoder–decoder cross-ablation.

DatasetEncoderDecoderExact ↑Edge F1 ↑
SBMLL0.00%73.86%
SBMLG0.00%93.12%
SBMGL6.25%96.81%
SBMGG90.63%99.95%
ProteinLL0.00%93.63%
ProteinLG57.03%99.83%
ProteinGL52.34%99.51%
ProteinGG93.75%99.86%

L: LG-Flow; G: GRAFT.

On Protein, replacing either component substantially improves reconstruction, while combining both achieves 93.75% exact reconstruction. On SBM, individual replacements improve edge-level accuracy, but the full GRAFT architecture achieves substantially higher exact reconstruction. These results demonstrate the complementary contributions of the relational encoder and decoder.

Thank you for highlighting the missing baselines and additional ablations. I will raise the score to 3.

AuthorsOpenReview ↗

Response to Follow-up Comments

We sincerely thank the reviewer for carefully reconsidering our rebuttal and additional experimental results. We greatly appreciate your constructive suggestions, which have helped us strengthen the baseline comparisons and better clarify the contributions of our relational encoder and decoder.

Thank you again for your time and thoughtful assessment.

Best regards,

The Authors

Reviewer_hwNEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents GRAFT, a novel method for latent graph generation. GRAFT reduces the common O(n2)O(n^2) complexity of previous latent graph generation method by setting node-wise relational node field as generation targets. Benchmark experiments on molecular, generic and protein graphs show excellent performance and efficiency optimization.

Critical strengths
  • This maker makes a very excellent novelty contribution that solves a key efficiency bottleneck of existing latent graph generation methods. Its proposed GRAFT method reduces computational complexity from O(n2)O(n^2) to O(n)O(n) by using relational node fields instead of pairwise node relations as generation targets.
  • Comprehensive experiments on generic, molecular and protein graph benchmarks are conducted. Strong experiment results demonstrate that the proposed GRAFT method achieves both good generation performance and computational efficiency.
Critical weaknesses
  • The key weakness is the lack of discussion about using flow matching as generation framework. The idea of using relational node fields as generation target is the key contribution of this paper, but it is unclear if flow matching is the best generation framework for it. Authors are encouraged to discuss the potential extension to other generation frameworks like VAE and diffusion.
Questions for authors

No other questions, see "Critical Weaknesses" section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer hwNE: Generative Framework and Future Directions

Thank you for your constructive and supportive review. We address your suggestion to clarify the choice of generative framework and potential extensions below.

(W1) Why Flow Matching, and Extensions to Diffusion or VAE-Based Generation

"it is unclear if flow matching is the best generation framework for it."

"Authors are encouraged to discuss the potential extension to other generation frameworks like VAE and diffusion."

Thank you for distinguishing the relational representation from the particular generative framework used to model it. Your suggestion prompted us to further examine the rationale behind our current design and its potential extension to other generative frameworks. We agree that relational node fields are not inherently tied to Flow Matching. Our choice is primarily motivated by two considerations: efficient latent transport and high-fidelity relational reconstruction.

(1) Why Flow Matching rather than Diffusion?

Because Stage-I maps discrete graphs into continuous node fields, Stage-II can in principle be instantiated with any continuous generative paradigm, including continuous-time score-based Diffusion. However, our Flow Matching formulation [1] adopts a straight-line conditional interpolation path with a constant target velocity:

Zt=(1−t)ϵ+tZ1,Z˙t=Z1−ϵ.Z_t=(1-t)\epsilon+tZ_1,\qquad \dot{Z}_t=Z_1-\epsilon.

This formulation is consistent with the straight-line transport principle emphasized by Rectified Flow [2]. In contrast, classical VP Diffusion [3] employs a time-dependent noising path:

Zt=αˉtZ1+1−αˉtϵ.Z_t=\sqrt{\bar{\alpha}_t}Z_1+\sqrt{1-\bar{\alpha}_t}\epsilon.

The linear conditional interpolation provides a simple velocity-prediction target, aligning with GRAFT's emphasis on efficient latent transport and low-step numerical integration. Moreover, it enables direct clean-endpoint estimation via Z^1(t)=Zt+(1−t)vθ(Zt,t)\hat Z_1(t)=Z_t+(1-t)v_\theta(Z_t,t), naturally supporting self-conditioning. More importantly, our theoretical sensitivity analysis (Proposition 1) shows that small perturbations in latent node fields can propagate through the relational decoder Jacobian and affect O(n2)\mathcal{O}(n^2) pairwise edge logits, making accurate endpoint prediction particularly important. By feeding back the estimated clean node field, self-conditioning promotes earlier stabilization of decoded relational structures, as supported by the reduced relational drift in Fig. 5(d) and the improved generation quality in Table 6.

(2) Why not a VAE?

Stage-I of GRAFT prioritizes high-fidelity reconstruction of graph relations. In contrast, a conventional VAE [4] regularizes its latent posterior toward a prior through a KL-divergence term, introducing a trade-off between reconstruction fidelity and latent regularity. This trade-off is particularly relevant for datasets with limited training samples, including SBM (128 graphs), Protein (128 graphs), and Point Clouds (26 graphs).

We also experimented with a VAE-based Stage-I, but obtained 0% exact graph reconstruction on both SBM and Protein (where exact reconstruction requires zero errors across all node pairs simultaneously). Consistent with Proposition 1, stochastic Gaussian perturbations and KL compression blur sharp decision boundaries in the node field, causing edge flips during pairwise decoding. These results suggest that, under our experimental settings, VAE-based representation learning struggles to preserve the relational structure required by GRAFT.

We therefore decouple relational representation learning from generative distribution modeling, allowing Stage-I to focus on structural reconstruction while Stage-II independently learns the distribution of relational node fields.

(3) Promising Future Direction.

Looking forward, bridging relational node fields with regularized latent spaces or alternative continuous generators remains a promising direction. Rather than imposing a strict standard Gaussian KL penalty, future work could explore lightweight latent regularization (e.g., a VAE with small scheduled KL weights, Wasserstein regularization, or joint Stage-I/II fine-tuning) to smooth the latent geometry for Flow or Diffusion transport without sacrificing exact edge reconstruction on small graph datasets. We have incorporated these directions into the Future Work discussion of the revised manuscript.

References

[1] Lipman, Y., et al. Flow Matching for Generative Modeling. ICLR, 2023.

[2] Liu, X., Gong, C., and Liu, Q. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. ICLR, 2023.

[3] Ho, J., Jain, A., and Abbeel, P. Denoising Diffusion Probabilistic Models. NeurIPS, 2020.

[4] Kingma, D. P., and Welling, M. Auto-Encoding Variational Bayes. ICLR, 2014.

Reviewer_hwNEOpenReview ↗

Follow-up Response

I apprecaite authors' efforts in rebuttal. My concerns have been addressed so I will keep my rate.

Dear Reviewer hwNE,

Thank you very much for taking the time to review our rebuttal! We appreciate your feedback as well as your reaffirmation of your rating.

Best regards,

The Authors

Reviewer_QAQrOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GRAFT, a latent graph diffusion model. This class of graph generative models differs from usual approaches in the space where diffusion occurs: classic (full-)graph diffusion occurs directly on the raw space of graphs, thus modelling explicitly the whole adjacency matrix and inheriting an O(n2)O(n^2) complexity; in contrast, latent graph diffusion models encode the graphs to a latent space composed of only node embeddings, denoise in that space, with complexity O(n)O(n) (alleviating the main limitation of current graph diffusion models), and finally decode back to the space of raw graphs. The approach itself has been proposed before (e.g., [1]), but GRAFT introduces two main innovations: i) a novel construction of node embeddings/latent representations that enables improved graph reconstruction (in particular, they solve failure cases -- exemplified with SBM and Protein -- of existing methods); ii) providing novel theoretical insights into relational error amplification and the role of self-conditioning in graph latent diffusion. In practice, GRAFT achieves competitive performance and improves upon existing latent graph diffusion models [1], but still lags behind the most performant full-graph diffusion models. Nevertheless, GRAFT demonstrates significantly shorter sampling time and reduced peak memory usage. Regarding reproducibility, the authors provide the theoretical and implementation details of their approach, as well as the source code. Overall, the paper is reproducible to a large extent, although a few aspects require further clarification (see "Questions for Authors"), which should be relatively straightforward to address.

Critical strengths
  1. GRAFT addresses the main limitation of graph diffusion models: their quadratic complexity in the number of nodes, which limits their scalability, while remaining competitive with methods that incur that quadratic complexity. While there is still room to improve, GRAFT is an important step towards establishing graph latent diffusion models.
  2. GRAFT achieves state-of-the-art latent graph diffusion: GRAFT solves the failure cases of LG-Flow [1], where the reconstruction rate is collapsed to 0. Moreover, GRAFT dominates LG-Flow in almost all angles: improved reconstruction rate for all datasets, faster sampling and reduced memory utilization; in terms of generative performance there are a few exceptions where LG-Flow slightly outperforms GRAFT, but always by small margins (while the opposite is not true).
Critical weaknesses
  1. The main motivation for reducing the complexity of graph diffusion models from quadratic to linear in the number of nodes is scalability, as current full-graph diffusion models are already extremely expressive for the range of number of nodes where they can be effectively deployed (see the benchmarks considered in the paper, which fall within that range, and where performance is nearly saturated for most of them). Even though I believe that showing that GRAFT achieves competitive performance on small-scale datasets (in terms of the number of nodes) is definitely relevant as a sanity check, getting results only in this regime does not fully illustrate the potential of GRAFT, as GRAFT is expected to be outperformed by full-graph models (which spend more compute) in this regime. Instead, GRAFT should be tested in the setting where it can become dominant: experiments on datasets containing larger graphs (> 1000 nodes), as considered in previous graph generation works [2-4], would strongly reinforce the message of the paper.
  2. The paper also lacks a concrete characterization of the trade-off between computational efficiency and generation quality. Given GRAFT's lower sampling time and memory requirements, it is relevant to investigate whether increasing the model capacity allows it to match or even surpass the performance of full-graph generative models under a matched inference-time budget (e.g.,sampling time or peak memory usage). Further increasing the number of function evaluations (NFEs) is another axis to scale inference budget. Such experiments would help characterize the efficiency–performance Pareto frontier and provide a more comprehensive assessment of the advantages enabled by GRAFT's node-only transport.

For these reasons, I believe the paper still lacks critical experimental evidence to fully establish the relevance of the proposed approach. This is further compounded by several questions regarding the completeness and clarity of the presentation, raised below in "Questions For Authors".

Questions for authors

Clarification Requests

  1. Throughout the paper, the authors refer to full-graph diffusion models complexity as O(n2)O(n^2), while assigning O(nd)O(nd) to the latent diffusion versions. Is there any particular reason to emphasize the dd in the complexity of the latent diffusion approach (since the full-graph diffusion also has that component)? If there's no particular reason for that difference, I would recommend to make both consistent (either include dd or leave it out in both).
  2. Reproducibility:
    1. The Stage-I training objective in Equation (5) is insufficiently specified. In particular, the paper does not explicitly define Lrelation\mathcal L_{\mathrm{relation}} and Laux\mathcal L_{\mathrm{aux}}, including their formulations and relative weighting. Could the authors provide these details?
    2. In Table 5(b), could the authors clarify what exactly is removed or modified in the “w/o Rel.” ablation? In particular, does it remove all pairwise and global relational processing, or only specific components of the relational encoder?
    3. In Figure 5(d), could the authors elaborate on how the relational drift is computed?
  3. Can the authors elaborate on where the advantage over LG-Flow in terms of sampling time and peak memory comes from? Can the model architecture (neural network) differences (extra parameters to accommodate larger latent node representation) explain the advantage or can we fully assign the improvement to the newly proposed encoding/decoding? Can you demonstrate this via ablation?
  4. Can the authors provide a per-stage runtime breakdown of GRAFT? Is there a bottleneck in any of the stages or are the loads evenly split?
  5. Figure 5 (a), (b): self-conditioning requires an extra no-grad forward pass at training time to extract the conditioning as input to the actual training step. This introduces additional computational overhead during training, which is not captured by the number of optimization steps. Therefore, using training steps on the xx-axis might be implicitly favouring self-conditioning. Is it possible to obtain the same results but using the wall-clock training time in xx-axis and elaborate on this?

Minor comments:

  • Figure 1: the yy axis label of the left plot is not fully readable.
  • In Section 3.1, it would be helpful to explicitly connect the operations described in the Relational Encoder and Relational Decoder paragraphs to the mappings EϕE_\phi and DψD_\psi, respectively
  • In Section 3.2: there is no previous definition of MM, NvalidN_\text{valid}, and vec⁡(.)\operatorname{vec}(.) operator. Please add these for full clarity of LtransportL_\text{transport}.

Suggestions for Future Work

  • Have the authors tried jointly training stage 1 and 2?
  • Have the authors looked into what do the trajectories look like (by decoding intermediate states)? Do the authors envision any way of how this information could be used to further push performance?

[1] - Principled Latent Diffusion for Graphs via Laplacian Autoencoders, Siraudin et al., arXiv 2026.

[2] - Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling, Chen et al., ICML 2023.

[3] - SparseDiff: Sparse Discrete Diffusion for Scalable Graph Generation, Qin et al., TMLR 2025.

[4] - Scalable Deep Generative Modeling for Sparse Graphs, Dai et al., ICML 2020.

9 comments
AuthorsOpenReview ↗

Response to Reviewer QAQr (Part 1): Large-Scale Graph Generation

Thank you for your constructive review and recognition of GRAFT’s strengths. We address your concerns about scalability, quality–efficiency trade-offs, and implementation details below.

(W1) Evaluation on Larger Graphs beyond 1,000 Nodes

"experiments on datasets containing larger graphs (> 1000 nodes)"

"getting results only in this regime does not fully illustrate the potential of GRAFT"

Thank you for this suggestion. We added Point Clouds experiments on graphs with 1,018–5,037 nodes to assess GRAFT's scalability and sampling efficiency.

Generation quality & scale coverage. In Table R1, GRAFT achieves a Ratio of 9.50, outperforming BiGG (38.2), SparseDiff (54.73), and EDGE (104.7), while trailing AutoGraph (3.0). Stage-I reconstruction reaches 2.33, indicating strong structural preservation. Increasing Stage-II Flow capacity improves the generation Ratio from 9.50 to 6.59 (a 30.6% improvement) at 100 NFEs (Table R3(b)).

For graph-size coverage, we generated 256 unconstrained samples using AutoGraph's official Point Clouds checkpoint. The largest contained 2,735 nodes; none reached 3,848 or 5,037 nodes (N/A in Table R2). GRAFT conditions on target size and generated graphs across all four sizes, up to 5,037 nodes. Using the same 256 node counts sampled by AutoGraph, the Ratios were 2.67 for AutoGraph and 8.52 for GRAFT at 100 NFEs.

Table R1: Point Clouds reconstruction and generation quality; lower is better.

MethodDeg. ↓Clus. ↓Orbit ↓Spec. ↓Ratio ↓
Training set0.00000.17680.00490.00431.0
BiGG0.09940.60350.36330.158938.2
EDGE0.44410.32981.07300.4006104.7
SparseDiff0.44651.16790.64830.108754.73
DiGressOOMOOMOOMOOMOOM
AutoGraph0.03070.3031‾\underline{0.3031}0.01670.0171‾\underline{0.0171}3.0
GRAFT†0.02510.20920.02060.00682.33
GRAFT‡0.0873‾\underline{0.0873}0.22920.1194‾\underline{0.1194}0.01229.50‾\underline{9.50}

† Stage-I reconstruction of input graphs; ‡ Stage-II generation of new graphs. Best and second-best generation results are bold and underlined, respectively; reconstruction and the training set are excluded. OOM: out of memory.

We follow the evaluation protocol of AutoGraph [5]. Stage-I reconstruction is excluded from generative ranking; bold and italics indicate the best and second-best generative results, respectively.

Sampling efficiency & Pareto trade-off. EDGE and BiGG offer fast inference or low memory but have higher Ratios (104.7 and 38.2). Compared with AutoGraph, GRAFT achieves 276–447× faster sampling and 5.3–5.9× lower peak GPU memory at the two graph sizes with available measurements (Table R2).

Table R2(a): Sampling time (s/graph) on Point Clouds. Lower is better.

Method1,018 nodes2,092 nodes3,848 nodes5,037 nodes
BiGG[1]12.5631.0970.61100.69
EDGE[2]0.800.971.902.66
SparseDiff[3]1109.534002.4113099.2637478.47
DiGress[4]OOMOOMOOMOOM
AutoGraph[5]137.96527.63N/AN/A
GRAFT0.501.183.015.31

Table R2(b): Peak GPU memory (GB) on Point Clouds. Lower is better.

Method1,018 nodes2,092 nodes3,848 nodes5,037 nodes
BiGG[1]0.530.530.530.53
EDGE[2]0.836.6427.3346.70
SparseDiff[3]1.657.3623.1923.76
DiGress[4]OOMOOMOOMOOM
AutoGraph[5]13.2532.02N/AN/A
GRAFT2.525.4312.6518.30

AutoGraph generates graph sizes autoregressively without explicit node-count conditioning. Other methods are evaluated at matched node counts for comparison. Bold and italics indicate the best and second-best results, respectively. OOM denotes out-of-memory failure. N/A indicates graph sizes not reached by AutoGraph in our 256 unconstrained samples.

References

[1] Dai, H., et al. Scalable Deep Generative Modeling for Sparse Graphs. ICML, 2020.

[2] Chen, X., et al. Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling. ICML, 2023.

[3] Qin, Y., et al. SparseDiff: Sparse Discrete Diffusion for Scalable Graph Generation. TMLR, 2025.

[4] Vignac, C., et al. DiGress: Discrete Denoising Diffusion for Graph Generation. ICLR, 2023.

[5] Chen, D., Krimmel, M., and Borgwardt, K. Flatten Graphs as Sequences: Transformers are Scalable Graph Generators. NeurIPS, 2025.

AuthorsOpenReview ↗

Response to Reviewer QAQr (Part 2): Quality–Efficiency Trade-offs

(W2) Quality–Efficiency Trade-offs across Inference Budgets

"the trade-off between computational efficiency and generation quality"

"increasing the model capacity" and "increasing the number of function evaluations (NFEs)"

We thank the reviewer for this suggestion. We evaluate NFE scaling, Stage-II Flow capacity, and full-graph baselines under approximately matched inference-time budgets.

(1) NFE Scaling: Trading Sampling Computation for Generation Quality

We evaluate 100, 500, and 1000 NFEs on MOSES, GuacaMol, Protein, and Point Clouds using the same checkpoints and evaluation protocols.

Table R3(a): Generation quality and inference time under different NFE budgets.

DatasetMetric100 NFE500 NFE1000 NFE
MOSESValidity ↑94.38%95.17%95.54%
FCD ↓0.79870.96880.9833
Time (s/graph) ↓0.01980.10000.1997
GuacaMolVUN ↑92.26%93.18%93.34%
FCD Score ↑84.0488.1688.85
Time (s/graph) ↓0.03240.10380.2051
ProteinRatio ↓5.103.2952.999
Time (s/graph) ↓0.05510.27570.5479
Point CloudsRatio ↓9.5008.9769.225
Time (s/graph) ↓1.0243.5346.651

Results. On Protein, increasing NFEs from 100 to 1000 reduces the Ratio from 5.10 to 2.999, a 41.2% improvement. On GuacaMol, the FCD Score increases from 84.04 to 88.85. Meanwhile, MOSES exhibits different trends in Validity and FCD, while Point Clouds achieves its best Ratio at 500 NFEs, indicating that different graph domains favor different inference budgets.

(2) Model Capacity Scaling: Increasing Stage-II Flow Capacity

On Protein and Point Clouds, we compare 8L/256 (9.70M parameters) and 12L/384 (32.37M parameters) at matched training steps and 100 NFEs.

Table R3(b): Generation quality under different Stage-II Flow capacities (Ratio ↓, 100 NFEs).

DatasetFlow ArchitectureParams100K200K300K400K600K
ProteinSmall (8L/256)9.70M13.598.596.085.705.10
Large (12L/384)32.37M6.554.733.483.392.66
Point CloudsSmall (8L/256)9.70M24.3021.5513.4212.369.50
Large (12L/384)32.37M23.8612.2210.258.226.59

Increasing Stage-II capacity consistently improves generation quality across both datasets. At 600K training steps, the larger Flow reduces the Ratio by 47.8% on Protein (5.10 → 2.66) and 30.6% on Point Clouds (9.50 → 6.59). The 2.33× Flow runtime difference in Table R4 quantifies the associated inference cost on Protein.

(3) Matched Inference-Time Comparison with Full-Graph Baselines

We compare against 100-step Cometh and DeFoG configurations, selecting GRAFT NFE budgets to approximately match their measured inference times.

Table R3(c): Molecular generation quality under approximately matched inference-time budgets on MOSES and GuacaMol.

DatasetMethodSteps / NFETime (s/graph) ↓Validity ↑VUN ↑FCD ↓
MOSESCometh [1]1000.0669885.28%82.12%1.280
GRAFT3500.0690195.14%86.73%0.923
MOSESDeFoG [2]1000.0848987.23%78.13%1.364
GRAFT4400.0863895.19%86.77%0.928
GuacaMolCometh [1]1000.1758293.43%92.64%1.674
DeFoG [2]1000.1886492.21%91.19%1.910
GRAFT6000.1749794.66%93.11%0.780

Both MOSES and GuacaMol use raw FCD (↓), distinct from the GuacaMol FCD Score (↑) reported in Table R3(a).

Under approximately matched inference-time budgets, GRAFT outperforms both Cometh and DeFoG across all three molecular generation metrics on both datasets.

On MOSES, the inference-time differences between matched configurations are within 3.1%, while GRAFT reduces FCD by 27.9% and 32.0% relative to Cometh and DeFoG, respectively. On GuacaMol, GRAFT achieves 53.4% and 59.2% lower FCD than Cometh and DeFoG, respectively, with no greater inference time, while also obtaining higher Validity and VUN.

GRAFT's lower per-step sampling cost allows more sampling steps within a comparable inference-time budget, yielding better generation quality in these comparisons.

The architectural mechanisms underlying these benefits, particularly asymmetric capacity allocation and single-pass relational decoding, are further analyzed in Q3 and Q4.

References

[1] Siraudin, A., Malliaros, F. D., and Morris, C. Cometh: A Continuous-Time Discrete-State Graph Diffusion Model. arXiv:2406.06449, 2024.

[2] Qin, Y., Madeira, M., Thanou, D., and Frossard, P. DeFoG: Discrete Flow Matching for Graph Generation. ICML, 2025.

AuthorsOpenReview ↗

Response to Reviewer QAQr (Part 3): Complexity and Method Details

(Q1) Consistent Complexity Notation and the Scope of Node-Only Transport

"I would recommend to make both consistent"

Thank you for pointing out the inconsistency. We have standardized the complexity notation to O(n2)\mathcal{O}(n^2) for dense full-graph diffusion and O(n)\mathcal{O}(n) for GRAFT, assuming fixed feature dimensions. We have also clarified in the appendix that these expressions describe the persistent generative state size, rather than the overall computational complexity.

(Q2.1) Stage-I Training Objective: Loss Definitions and Relative Weights

"The Stage-I training objective in Equation (5) is insufficiently specified."

"including their formulations and relative weighting."

We thank the reviewer for pointing out this omission. The Stage-I objective is LStage−I=Lrelation+Laux\mathcal L_{\mathrm{Stage-I}}=\mathcal L_{\mathrm{relation}}+\mathcal L_{\mathrm{aux}}, where the relation reconstruction loss is the cross-entropy averaged over valid node pairs:

Lrelation=−1∣P∣∑(i,j)∈Plog⁡pθ(yij∣Z).\mathcal L_{\mathrm{relation}}=-\frac{1}{|\mathcal P|}\sum_{(i,j)\in\mathcal P}\log p_\theta(y_{ij}\mid Z).

Here, P\mathcal P denotes the set of valid node pairs, and yijy_{ij} denotes the relation label for pair (i,j)(i,j).

For molecular datasets, the auxiliary loss is defined as

Laux=λaLatom+λcLcharge+λhLH+λrLaromatic.\mathcal L_{\mathrm{aux}}=\lambda_a\mathcal L_{\mathrm{atom}}+\lambda_c\mathcal L_{\mathrm{charge}}+\lambda_h\mathcal L_{\mathrm H}+\lambda_r\mathcal L_{\mathrm{aromatic}}.

Each component is a cross-entropy loss averaged over valid atoms. For MOSES, all four loss weights are set to 1. For GuacaMol, the atom loss has weight 1, while the charge, hydrogen, and aromaticity losses each have weight 1/3. For non-molecular datasets, no auxiliary loss is applied.

We have clarified these definitions and weights in the revised manuscript.

(Q2.2) Exact Definition of the “w/o Rel.” Ablation

"does it remove all pairwise and global relational processing, or only specific components of the relational encoder?"

Thank you for asking us to specify the intervention precisely. We replace node-specific relational aggregation with graph-level aggregation.

In the full model, each node receives its own relational summary mi(ℓ)=Pool⁡jπij(ℓ)m_i^{(\ell)}=\operatorname{Pool}_j\pi_{ij}^{(\ell)}. In the ablation, pairwise information is instead aggregated into a shared graph-level representation, without providing node-specific relational messages to individual nodes.

Thus, pairwise computation and global aggregation are retained, while node-specific relational aggregation is removed. We have clarified this distinction in the main-text discussion accompanying Table 5(b).

(Q2.3) Definition and Computation of Relational Drift

"In Figure 5(d), could the authors elaborate on how the relational drift is computed?"

Thank you for the question. Figure 5(d) measures the relational discrepancy between the predicted endpoint and the final generated latent along the same Flow trajectory, defined as

ER(t)=1n(n−1)∑i≠j[sij(Z^1(t))−sij(Zfinal)]2.E_R(t)=\frac{1}{n(n-1)}\sum_{i\neq j}\left[s_{ij}(\hat Z_1(t))-s_{ij}(Z_{\mathrm{final}})\right]^2.

Here, nn denotes the number of valid nodes. The first term represents the relational predictions from the estimated endpoint at transport time t, while the second corresponds to the final latent along the same ODE trajectory. The pairwise scores are edge-existence logits produced by the frozen Stage-I decoder. Padding nodes and diagonal entries are excluded, and results are averaged over 512 MOSES samples.

Connection to our theoretical analysis. Eq. (10) characterizes how latent perturbations affect decoded relations through the decoder Jacobian, while Proposition 1 describes their potential amplification. The lower relational discrepancy with self-conditioning in Figure 5(d) indicates earlier stabilization of the predicted relational structure, consistent with the generation-quality improvements in Table 6. We have clarified this connection in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.