acceptodds
Under review as a conference paper at ICLR 2027

GraViti: Graph-Level VAE with Relaxed Permutation Invariance

Abstract

We introduce GraViti, a transformer-based graph-level variational autoencoder that encodes entire graphs into single, fixed-dimensional latent vectors rather than per-node embeddings, yielding a graph-level latent space that supports smooth interpolation, property-guided search, and other downstream tasks beyond the reach of node-level approaches. GraViti achieves state-of-the-art reconstruction accuracy on large molecular graph datasets while offering competitive single-step generative performance. Our central finding concerns where permutation invariance is needed in such a model, and where it is not. The encoder remains permutation-invariant throughout, ensuring the model generalizes to graphs regardless of input order. The reconstruction loss, however, does not need to be: when training data is provided in a consistent node order, comparing predictions to targets directly in that order, without a graph-matching step, is sufficient. Relaxing invariance in the loss alone lets GraViti avoid the cubic-cost matching step required by prior graph-level autoencoders, reducing complexity to quadratic while improving reconstruction fidelity. We show the resulting latent space is chemically meaningful, supporting controlled molecular editing and property optimization, recovering established chemical trends, and enabling direct regression of graph-level properties from latent embeddings.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Neural Networks, Graph Representations, Graph Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_m1siOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a novel transformer-based graph-level variational autoencoder that encodes an entire graph into a single latent representation instead of a node-level latent representation, as in existing methods. The encoder creates a global latent representation, which is thus invariant to input graph ordering. The central paper claim is that when training data is provided with consistent node ordering, the authors can compare predictions directly to targets in that ordering without sorting to solve graph matching problems. By doing so, the authors can avoid cubic complexity in the matching step. The authors mainly tested their method on molecular graph editing, property optimization, and molecular graph generation tasks.

Critical strengths

The authors raised an interesting question on latent graph generative modeling. Indeed, there is a bottleneck when we model graphs with a global latent representation, which requires solving a graph matching problem when computing reconstruction losses. In addition, the authors showed some promising results on different molecular tasks from their learned latent graph representations.

Critical weaknesses
  • One of the main weaknesses of the paper is the lack of related work. The prior work, GLAD [1], explicitly discussed different approaches to modeling graphs in latent space, from latent nodes to latent graphs, and both discrete and continuous representations. Especially, the authors in GLAD showed that their global latent representation did not work well in latent-node cases in the context of graph generation. In addition, they also discussed graph matching problems of global graph latent representations. Thus, I would suggest the authors put analysis compared to GLAD, since I feel this is the most related work to this paper.

  • Related to the GLAD paper, it's not very clear how the proposed method can avoid the latent node collapsing problem in GLAD. In Section 3, when decoding from graph-level to node-level representation, the authors simply duplicate the graph-level latent representation, which contributes to the latent node collapsing problem, where different local neighborhood node encodings share a similar latent representation. This causes difficulty when reconstructing original graphs.

  • The authors claim that their model is scalable. I think they make such a claim because they can avoid solving the cubic graph matching problem. However, latent-node representations can also avoid this problem by keeping node ordering information from encoder input to decoder output. Thus, I would suggest that the authors revise this claim. Especially, the claim remains very generic, given that their graph transformer has quadratic complexity in the number of nodes compared to less expensive standard message-passing graph neural networks.

  • Another claim on relaxed permutation invariance remains ambiguous. I still can not fully understand how the authors reconstruct original graphs while the node ordering information is collapsed/destroyed by their global latent node representation without sorting graph matching problems. There is no such clear analysis of relaxed permutation invariance in their main methodology section (Section 3). In addition, latent graph modeling has been a well-studied subject for a long time; the main contribution here remains quite trivial.

  • The authors emphasized the necessity of predicting graph size, but there is no ablation study to show that if the prediction helps or we simply samples from empirical graph size distributions.

  • On the graph generation task, even though the authors acknowledge that this is not the main focus of their study. However, I would think this is an utmost important task; through this, we can assert how well the model can capture data distributions. There are many state-of-the-art baselines on this task [1,3,4]; I would recommend that the authors compare with them to make their model look stronger and more promising.

Questions for authors
  1. How does the model compare to GraphVAE [2], which also uses a global latent graph representation?

  2. How does the model overcome the latent-node collapsing problem in GLAD, where all latent nodes share the same representation? Because broadcasting from the latent graph representation to all latent node ones create the same latent representation for all nodes, while it should not be.

  3. How does the model handle different node input orderings? Since, for the current model, each node ordering causes a different loss objective, which imposes instability in training and model generalization.

  4. What does it mean the stable node ordering is available (in the Conclusion)?

  5. How do the authors sample the number of nodes for the generation task? Is the number of nodes predicted by the model as shown in Section 3? In this case, how would the model performance compare to the scenario when we simply sample the number of nodes from empirical distributions?

References:

[1] GLAD: Improving Latent Graph Generative Modeling with Simple Quantization, AAAI 2025. Nguyen et al.

[2] GraphVAE: Towards generation of small graphs using variational autoencoders, arXiv:1802.03480v1. Simonovsky et al.

[3] DiGress: Discrete Denoising diffusion for graph generation, ICLR 2023. Vignac et al.

[4] Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations, ICML 2022. Jo et al.

9 comments
AuthorsOpenReview ↗

Rebuttal: Reviewer m1si

We thank the reviewer for their thorough and extensive review. We answer below the different weaknesses and questions that arise throughout the review, and hope that our answer and results will clarify several of the points raised.

W1

One of the main weaknesses of the paper is the lack of related work. The prior work, GLAD [1], explicitly discussed different approaches to modeling graphs in latent space, from latent nodes to latent graphs, and both discrete and continuous representations. Especially, the authors in GLAD showed that their global latent representation did not work well in latent-node cases in the context of graph generation. In addition, they also discussed graph matching problems of global graph latent representations. Thus, I would suggest the authors put analysis compared to GLAD, since I feel this is the most related work to this paper.

We thank the reviewer for pointing us to GLAD [1], which we agree is related to the domain of graph autoencoding, and which we will discuss explicitly in the revision.

Different latent spaces. GLAD operates in a node-level latent space: its encoder produces a set of node embeddings Zraw={zi}i=1NZ_{\mathrm{raw}} = \{z_i\}_{i=1}^{N}, each quantized independently. The latent is therefore size-dependent and permutation-equivariant. Ours is a graph-level, permutation-invariant latent.

Why GLAD avoids graph-level latents. The GLAD authors motivate their choice by noting that graph-level latents require solving a graph-matching problem in the reconstruction loss, and that "this extra step adds more computational complexity and is inaccurate for large graphs" ([1], Sec. 2, "Continuous-graph latent space"). This matches our findings in Table 1, where the reconstruction quality of matching-based graph-level models degrades as graph size grows. Our approach is designed to avoid the matching step: we use a canonical node ordering for the training targets and the reconstruction loss. As a result, our AE reaches on PubChem32 (molecules of size 32) a reconstruction accuracy of 98%, compared to 73.3% for GRALE, the state-of-the-art matching-based model, a gap of 24.7 points.

GLAD's graph-level baseline. GLAD includes a graph-level model as a baseline, described only as a "vanilla VAE on a global structure, pooled latent nodes, to get a continuous-latent representation" [1]. Neither the paper nor the released code details its architecture or training, but we suppose that it was trained using matching (as stated in Section 2). That model is also simply evoked in an experiment but never thourouhly analyzed by the authors.

Planned revisions. We will cite GLAD in the introduction and related work (node-level latent models) and when motivating our matching-free design.

AuthorsOpenReview ↗

Rebuttal: Reviewer m1si - II

W2 / Q2

Related to the GLAD paper, it's not very clear how the proposed method can avoid the latent node collapsing problem in GLAD. In Section 3, when decoding from graph-level to node-level representation, the authors simply duplicate the graph-level latent representation, which contributes to the latent node collapsing problem, where different local neighborhood node encodings share a similar latent representation. This causes difficulty when reconstructing original graphs.

How does the model overcome the latent-node collapsing problem in GLAD, where all latent nodes share the same representation? Because broadcasting from the latent graph representation to all latent node ones create the same latent representation for all nodes, while it should not be.

We thank the reviewer for this question. We agree that broadcasting a single vector to nn nodes would yield identical node inputs if nothing else were added. We explain below why that collapse would not be a failure mode for our model, and why the broadcast does not lead to such a collapse.

Where collapse arises. Latent-node collapse is a failure mode of node-level latents: each node latent must encode its own local neighborhood, and several latents can converge to similar vectors, preventing reconstruction of diverse local structures, because the node-level latents are then propagated through permutation-equivariant layers (so identical rows in the input lead to identical rows in the output). In our model, the latent is a single graph-level vector zz encoding the whole molecule. High-level node-level representations appear only at two intermediate points (Fig. 1):

  • Encoder side: XrepX_{\mathrm{rep}} (between blocks (A) and (B) in Figure 1). Rows of XrepX_{\mathrm{rep}} (one per node) could in principle become similar, for example through oversmoothing in the graph updates of block (A). However, XrepX_{\mathrm{rep}} is not a latent, and it is never propagated through a permutation-equivariant layer directly. Its only role is to be pooled by the permutation-invariant readout into the global graph representation zz, and the encoder is trained end-to-end with the reconstruction loss (meaning that, unlike in GLAD, no node-level latent has to encode its own neighborhood and collapse is only penalized if it discards graph-level information). Therefore, a collapse at the XrepX_{rep} level is not incompatible with a good graph-level representation; if reconstruction performance remains high (as we observe with GraViti), it simply indicates that the graph-update part of the encoder is powerful enough to summarize the entire graph into a rank-one matrix before readout while conserving all information relevant for reconstruction. We provide an ablation study where we replace the transformer pooling by a simple sum pooling (see the updated Table 4). Reconstruction accuracy drops from 99.5% to 99.4%, showing that XrepX_{rep} remains informative (but saying nothing about collapse). To further investigate whether this phenomenon occurs, we compare the average cosine similarity between node representations belonging to the same graph with the average cosine similarity across all node pairs. Across 1024 randomly sampled molecules from PubChem16, we obtain an average within-graph cosine similarity of 0.044, compared to 0.057 across all node pairs. This suggests that node representations within individual graphs are, if anything, slightly more diverse than the overall population of node representations. In particular, we find no evidence that nodes belonging to the same graph systematically converge to similar representations. While this analysis cannot rule out all forms of representation collapse, it provides no indication of within-graph collapse in XrepX_{\mathrm{rep}}.

  • Decoder side: duplication of zz. We duplicate zz nn times (nn being the predicted size) and combine it with size information and a sinusoidal positional encoding SPE\mathrm{SPE}. The nn node inputs thus share the same content, coming from zz, and differ only by position. This is deliberate. Because the training targets and reconstruction loss use a canonical node ordering (see W1), position ii always refers to the ii-th node in that order, so the decoder can learn to produce node-specific outputs from the shared content and the position. Without SPE\mathrm{SPE} all node inputs would be identical and reconstruction would fail.

Summary. The graph-level design removes the need for node latents to encode local neighborhoods, which is the source of collapse in GLAD. The broadcast is made safe by SPE\mathrm{SPE} together with the canonical ordering, and the high reconstruction accuracy in Table 1 indicates that nodes are well distinguished in practice.

AuthorsOpenReview ↗

Rebuttal: Reviewer m1si - III

W3

The authors claim that their model is scalable. I think they make such a claim because they can avoid solving the cubic graph matching problem. However, latent-node representations can also avoid this problem by keeping node ordering information from encoder input to decoder output. Thus, I would suggest that the authors revise this claim. Especially, the claim remains very generic, given that their graph transformer has quadratic complexity in the number of nodes compared to less expensive standard message-passing graph neural networks.

We thank the reviewer for this remark and agree that the claim should be made more precise. Our scalability claim has two parts, and neither is meant as a claim of efficiency relative to message-passing GNNs or node-level models:

  • Computational scalability relative to matching-based graph-level models. As the reviewer points out, this advantage stems from avoiding the costly (cubic) matching step. It holds only against graph-level models, whose standard is to use a matching step: node-level models need no matching, since each node latent stays tied to its input node and output node by construction. We therefore claim no scalability advantage over node-level VAEs. We make the trade-off (scalability vs. graph-level expressivity) explicit at the end of Section 2; and we compare in Appendix A against a node-level variant of our architecture (identical except for the absence of the readout/readin). That variant expectedly reconstructs better, but at the cost of losing the graph-level representation. We made it explicit in the revised version that our scalability claim holds with regards to matching-based graph-level methods, which is the core claim of our paper.
  • Reconstruction quality as graphs grow. Still at the graph level, our canonical-ordering approach reaches better reconstruction than matching-based approaches. Matching thus acts as a performance bottleneck that becomes harder to train as graphs grow. GLAD makes the same observation in Section 2: "Graph-matching scales poorly and is inaccurate for large graphs."

To support this claim, we report intermediate results for two models of the same size (35M parameters, both autoencoder and using aromatic form of PubChem32, our largest dataset) trained for 5 epochs on the same computational architecture: GraViti reaches 73.5% after a single epoch (40.5 h), which GRALE only matches after five (73.3% at 229.8 h), about 5.7× less training time for the same accuracy. Then, our model keeps improving to reach 97.5% in five epochs. We will add these results in the revised paper. This strongly support our claim that our approach alleviates the computational and performance bottleneck induced by the matching step by leveraging our aligned loss.

Table: Accuracy vs. cumulative training time (GraViti vs. GRALE), AEs (35M parameters), PubChem32.

EpochGraViti Acc.GraViti Time (h)GRALE Acc.GRALE Time (h)Δ Acc. (pts)
173.5%40.548.8%56.0+24.7
286.8%76.361.2%98.7+25.6
392.7%113.767.9%143.5+24.8
496.2%149.471.1%185.4+25.1
597.5%186.573.3%229.8+24.2

Table: Compute-matched comparison.

GraVitiGRALE
Avg. time per epoch37.3 h46.0 h
Time to reach ~73% accuracy40.5 h (epoch 1)229.8 h (epoch 5)
Accuracy at ~186 h97.5% (epoch 5)71.1% (epoch 4)

On the quadratic cost of the transformer. We agree that self-attention is O(n2)O(n^2) in the number of nodes, whereas standard message-passing GNNs are O(∣E∣)O(|E|), roughly linear for molecular graphs. Our scalability claim concerns the matching step, which costs O(n3)O(n^3) and comes on top of the backbone. While using message passing GNNs is tempting, it suffers from two drawbacks:

  • an MPNN-based encoder degrades the performance of the model in a significant manner (the ablation results are shown in Table 3);
  • the decoder still has to predict the full adjacency matrix, which is already O(n2)O(n^2), so attention does not change its asymptotic cost. Neither MPNNs nor node-level autoencoders alleviate this issue, since the bottleneck is predicting the adjacency matrix, which they all need to do.

In the revised paper we have (i) replaced the generic "scalable" with more detailed statements, (ii) added the supporting GLAD citation, (iii) clarified that our method is quadratic through the use of attention and the need to reconstruct the entire adjacency matrix and (iv) will enrich the training time table with more complete values.

Reviewer_YLuAOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces GraViti, a graph variational autoencoder that maps variable-sized graphs into a fixed-dimensional latent space using a graph transformer and then decodes them back into graphs. The main contributions are a novel architecture (with many interesting components, e.g., the graph size prediction module) and a permutation-sensitive reconstruction loss that improves performance while being substantially more scalable than matching-based objectives.

Critical strengths
  • The paper is very well-written and easy to follow. The model architecture is clear, well-motivated, and sound.
  • The paper presents a wide set of experiments that support the benefit of a graph-level VAE as well as the new reconstruction loss.
Critical weaknesses

None, only minor issues (see questions)

Questions for authors
  1. The used font does not match the officially recommended one.
  2. L133: "at no extra cost" is imprecise; there is still some cost associated with computing the encodings, and by increasing the dimensionality of the features, the cost associated with the model forward pass also increases
  3. L138-142: The padding for batching is an implementation detail and distracts from the main flow of the section
  4. Node-level VAEs are missing from the experiments. The paper would be strengthened by adding them for completeness.
  5. In Table 5, why is CFM missing on PubChem?
2 comments
AuthorsOpenReview ↗

Rebuttal: Reviewer YluA

We first of all thank reviewer YluA for their review, as well as for pointing out several points where our paper's clarity was improved. We will upload shortly a reviewed paper where the changes were made. We adress their five questions below:

The used font does not match the officially recommended one.

  • Thank you for reporting the font issue. The problem has been fixed. We would like to note that the font used in the submitted PDF was actually larger than the official template font. As a result, the original submission would not have violated the page limit had the font been rendered correctly.

L133: "at no extra cost" is imprecise; there is still some cost associated with computing the encodings, and by increasing the dimensionality of the features, the cost associated with the model forward pass also increases

  • We thank the reviewer for their observation. It is true that "no extra cost" is ambiguous. What we meant was that there is neither outside information required nor complex algorithms applied (such as cycle counting), but a simple column-wise summation. We also observe in the ablations that this vector barely affects the performance. On the other hand, it is true that this introduces extra layer blocks (and therefore parameters). We have removed the claim about added costs and instead write the following in our revised manuscript: "These inject global structural information into the graph transformer."

L138-142: The padding for batching is an implementation detail and distracts from the main flow of the section

The paragraph in question was moved to Appendix C (Hyperparameters and training details), and the main text now flows naturally.

Node-level VAEs are missing from the experiments. The paper would be strengthened by adding them for completeness.

The main objective of our approach is to show that a loss based on a consistent canonical ordering allows to improve the performance of graph-level VAEs rather than being a destructive bias. Node-level VAEs skip this problem altogether by not being permutation-invariant, and therefore training through an aligned loss function. We therefore expect them to outperform our approach on reconstruction tasks due to training being easier.

We illustrate this in Appendix A, where we compare our architecture to a node-level GraViti, where reconstruction performance is significantly better, at the cost of sacrificing the graph-level latent space (and therefore interpolation/optimization with varying size). We also observe that this leads to degraded generation, which is consistent with the literature on node-level VAE sampling.

In Table 5, why is CFM missing on PubChem?

Graph generation is not the main focus of our paper and serves primarily as an additional diagnostic of the structure of the learned latent space. We therefore do not expect to outperform methods specifically designed for graph generation. We include the CFM baseline mainly as a point of reference to illustrate the performance of a recent graph-generation approaches.

Since the original CFM paper does not report results on PubChem, including this benchmark would require retraining CFM ourselves. The authors report approximately two days of training on QM9, while PubChem contains 120 times more molecules, making such an effort computationally expensive. Given that the purpose of this experiment is not to establish a new state of the art in graph generation but rather to provide a qualitative assessment of the latent space, we did not consider the additional computational cost of reproducing CFM on PubChem to be justified. We will clarify this point in the revised manuscript.

Conclusion

We thank again the reviewer for their relevant comments, and remain at their disposal should any further question arise during this discussion phase.

Thank you for the response; it clarifies my questions, and I maintain my score and my recommendation to accept this paper.

Reviewer_ufo1OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

GraViti is a Graph-VAE method that uses node embeddings, edge embeddings and the adjacency matrix to map into a single-vector representation space. The method can be used to leverage desired properties of VAEs such as smooth interpolation and property guided search. GraViti outperforms the baselines on molecule datasets.

Critical strengths

I could not find one.

Critical weaknesses
  1. The authors claim that graph-level autoencoders are underexplored. But in reality, there are several works that do this: “SE-VGAE: Unsupervised disentangled representation learning for interpretable architectural layout design graph generation”, “BrepVGAE: Variational Graph Autoencoder with Unified Latent Representation for B-rep”, “Unifying Generation and Prediction on Graphs with Latent Graph Diffusion”, “NeVAE: A Deep Generative Model for Molecular Graphs”.

  2. Table 9 reports the train val and test split sizes. The size of train split for PubChem16 and PubChem32 datasets are 97.36% and 99.78%. For the Coloring dataset the size of the train split is 93.75%. These choices are very skewed. More importantly, the hyperparameters often differ across datasets.

  3. As outlined in weakness 1, the authors should compare to other works that propose graph-autoencoders. Moreover, diffusion and flow based approaches should also be considered for the generation task.

  4. The overall block diagaram is very poor. What is "readin"?

Questions for authors
  1. Can you explain why all methods have 100% accuracy for ASIA-BN dataset? This is a very surprising result. All methods seem to have a very high reconstruction accuracy on most datasets.

  2. Can you provide comparison with some other baselines for both the reconstruction task and generation task.

  3. The ablation study in Table 3 is inconclusive. It does not provide clarity on the contribution of the various components of the pipeline. Can the authors provide clarity on this?

  4. GRALE’s PubChem32 accuracy is 73.3 in Table 1 (“trained until convergence”), Table 2 and 10. The edit distance is also 0.70 in both Table 1 and Table 10. GRALE’s numbers in PubChem16 also change between tables (96.9 vs. 94.6). Can you check this?

4 comments
AuthorsOpenReview ↗

Rebuttal: Reviewer ufo1

We thank the reviewer for their multiple comments. We will clarify the parts they point as unclear and will answer their question in the rebuttal below.

W1/W3 - Q2

The authors claim that graph-level autoencoders are underexplored. But in reality, there are several works that do this: “SE-VGAE: Unsupervised disentangled representation learning for interpretable architectural layout design graph generation”, “BrepVGAE: Variational Graph Autoencoder with Unified Latent Representation for B-rep”, “Unifying Generation and Prediction on Graphs with Latent Graph Diffusion”, “NeVAE: A Deep Generative Model for Molecular Graphs”.

As outlined in weakness 1, the authors should compare to other works that propose graph-autoencoders.

Can you provide comparison with some other baselines for both the reconstruction task and generation task.

We thank the reviewer for these references. First of all, we would like to point out that two of them are actually node-level approaches, and therefore do not suffer from the matching problem that our approach was explicitly designed to address. In particular,

  • the description of the latent in part 4.1 of "Unifying Generation and Prediction on Graphs with Latent Graph Diffusion" explains that the latent is composed of both node and edge representations: "In this latent space, all n nodes and n2n^2 “augmented edges” have continuous latent representations."
  • Section 3 of "NeVAE: A Deep Generative Model for Molecular Graphs": "In the above characterization, note that we define one latent variable per node, i.e., we have a node-based latent representation, and the number of nodes is a random variable. As a consequence, both the latent representation as well as the graph can vary in size."

Regarding SE-VGAE and BrepVGAE, they are indeed relevant graph-level autoencoders and we have added both to the related work of our revised manuscript. Neither of these domain-specific work addresses the central question studied in GraViti, namely how permutation invariance should be handled in general graph-level reconstruction and comparing matching-based approaches to matchless ones. SE-VGAE focuses on disentangled representation learning for architectural floor-plan generation, while BrepVGAE targets a distinct application domain. For completeness, BrepVGAE was published on September 17, 2026, one day before the ICLR 2027 abstract deadline. The ICLR 2027 Reviewer Guide states that "We consider papers contemporaneous if they are published within the last two months.", emphasizing the interest in the field.

We will clarify the "under-explored" claim by making it explicit that we mean that relative to the node-level autoencoders (we will also cite the two node-level papers you kindly provided). We will furthermore cite both graph-level approaches and situate our approach with regards to them.

W2

Table 9 reports the train val and test split sizes. The size of train split for PubChem16 and PubChem32 datasets are 97.36% and 99.78%. For the Coloring dataset the size of the train split is 93.75%. These choices are very skewed. More importantly, the hyperparameters often differ across datasets.

The hyperparameters differ across datasets because the datasets vary substantially in size and complexity. For example, PubChem32 requires significantly more representational capacity than PubChem16. Similar dataset-specific hyperparameter choices are common in prior work, including GRALE.

Regarding the split sizes, they were initially chosen to obtain practical dataset sizes (approximately 12.5M training samples and 40k validation samples for PubChem16, and approximately 80k validation samples for PubChem32). These choices were made for computational convenience and were not optimized to improve performance. Importantly, all methods, including GraViti and all baselines, were trained and evaluated on exactly the same splits. Therefore, the relative comparison remains fair and unaffected by the specific train/validation/test proportions.

Furthermore, GraViti consistently outperforms GRALE by a substantial margin (up to 24.7%), and the datasets are large (several millions molecules), so that the observed improvements cannot be explained by minor variations in split ratios alone. Finally, the complete dataset generation and splitting procedure is publicly available in the supplementary code, ensuring full reproducibility and transparency

AuthorsOpenReview ↗

Rebuttal: Reviewer ufo1 - II

W3-2

Moreover, diffusion and flow based approaches should also be considered for the generation task.

We thank the reviewer for this comment. GraViti is primarily a graph-level autoencoder, and our evaluation focuses on reconstruction, latent-space structure (interpolation, optimization, denoising), and downstream prediction. We agree, however, that generation provides a useful diagnostic of how well the latent space captures the underlying data distribution. Nonetheless, we point that GraViti is not built to be a competitive molecular generation model (we point the reviewer to the "Graph generation capability") paragraph in section 5, which details why graph VAEs are consistently not competitive with state-of-the-art generation methods or molecule-specific generation methods.

We nonetheless provide comparison to a recent specialized few-steps flow-based approach, as requested by the reviewer (Categorical flow maps). On QM9, the resulting GraViti + latent flow pipeline achieves generation quality comparable to Categorical Flow Maps (CFM, whose results we report directly from the original paper):

QM9StepsValid. (%)Uniq. (%)VUN (%)FCD
GraViti + latent FM194.199.293.31.66
CFM ECLD195.889.285.52.16
CFM CSD190.091.882.62.14
GraViti + latent FM295.699.895.40.75
CFM ECLD297.096.693.70.52
CFM CSD291.197.688.90.49

While GraViti was not designed as a state-of-the-art molecular generator, the results in Appendix L suggest that the learned graph-level latent representation captures meaningful aspects of the underlying data distribution. In particular, a lightweight latent flow trained on frozen embeddings substantially improves sampling quality, supporting the claim that the latent space itself is well structured.

W4

The overall block diagaram is very poor. What is "readin"?

The "readin" block is the symmetric counterpart of the "readout" block, since the readin block goes from graph-level to node-level, while the readout block does the opposite. We understand that this can be confusing. We have updated the main text to prevent any ambiguity. We also updated the figure's caption to make it clearer.

Q1

Can you explain why all methods have 100% accuracy for ASIA-BN dataset? This is a very surprising result. All methods seem to have a very high reconstruction accuracy on most datasets.

The fact that GRALE perfectly reconstructs ASIA-BN is consistent with its strong performance on small datasets, where the detrimental effect of the matching procedure is limited. GraViti also achieves perfect reconstruction on this dataset. This is due to the particular structure of ASIA-BN, which consists of directed acyclic graphs (DAGs). In such graphs, a natural partial ordering of the nodes exists, making it easier to obtain a consistent canonical ordering across samples. Since GraViti relies on consistent canonical orderings during training, ASIA-BN provides a particularly favorable setting. More broadly, this result suggests that even a partial or semi-ordering, when sufficiently consistent across graphs, can be enough for GraViti to learn accurate reconstructions. We will clarify this point in the main text.

Q3

The ablation study in Table 3 is inconclusive. It does not provide clarity on the contribution of the various components of the pipeline. Can the authors provide clarity on this?

The main point of Table 3 is to demonstrate that the performance of GraViti does not stem from architectural improvements (since it is remarkably robust to changes) with the main exception being that GNN-based encoders lose significantly to transformer-based ones.

This reinforces the fact that our gain in performance comes not from our model's architecture but from the design choice to avoid the matching step (which is a core takeaway of our paper). This is further shown by the fact that, when we enforce a matching-based loss function, GraViti's performance drops down to that of GRALE.

AuthorsOpenReview ↗

Rebuttal: Reviewer ufo1 - III

Q4

GRALE’s PubChem32 accuracy is 73.3 in Table 1 (“trained until convergence”), Table 2 and 10. The edit distance is also 0.70 in both Table 1 and Table 10. GRALE’s numbers in PubChem16 also change between tables (96.9 vs. 94.6). Can you check this?

Table 2 and 10. The edit distance is also 0.70 in both Table 1 and Table 10. GRALE's numbers in PubChem16 also change between tables (96.9 vs. 94.6). Can you check this?

We thank the reviewer for the careful check.

Tables 1 and 10 (PubChem32). The reviewer is right. GRALE's PubChem32 entries in Table 1 (73.3%, 0.70) are the aromatic results of Table 10, copied into the kekulized table by mistake. We apologize for this mistake, which affects only this one entry (PubChem32 for GRALE on kekulized molecules). Since kekulized and aromatic forms are highly redundant (they encode the same molecules) and give very close reconstruction results wherever both were run, we will make the aromatic form the main table, so that all entries come from one setting for fair comparison, while the kekulized table will be moved to the appendix. The one problematic configuration (kekulized PubChem32 for GRALE) is being retrained, and will be added to the table in the appendix once done. After 3 epochs, it has reached an accuracy of 68.5%, which is very close to the 67.9% obtained with aromatic bonds at the same epoch. These preliminary results are consistent with our observation that the two representations yield similar reconstruction accuracies. The main text has been edited wherever necessary to reflect that editing. The conclusion is unchanged and GraViti still outperforms GRALE by 24.7 points on PubChem32, which is a large margin.

Table 2 (PubChem16 and PubChem32). Table 2 compares the AE versions of GRALE and GraViti in aromatic form at a matched budget of 5 epochs and comparable size (7M parameters on PubChem16, 35M on PubChem32), to compare training time. Table 10 instead reports each model's best result, after hyperparameter tuning and clear convergence, which is why the numbers differ. On PubChem32, GRALE's 73.3% at 35M parameters means it has already converged after 5 epochs.

This is visible in the following table, which we will add to the revised paper, which actually makes GraViti's scalability claim all the clearer:

Table: Accuracy vs. cumulative training time (GraViti AE vs. GRALE), PubChem32, aromatic form.

EpochGraViti Acc.GraViti Time (h)GRALE Acc.GRALE Time (h)Δ Acc. (pts)
173.5%40.548.8%56.0+24.7
286.8%76.361.2%98.7+25.6
392.7%113.767.9%143.5+24.8
496.2%149.471.1%185.4+25.1
597.5%186.573.3%229.8+24.2

At this budget GraViti is both more accurate (97.5% vs. 73.3%) and faster (186.5 h vs. 229.8 h). Since GraViti is not bottlenecked by matching, it keeps improving with more training (98.0% in Table 10), while GRALE does not. We will state the budget (epochs and parameters), model variant and bond form in the captions of Tables 1, 2 and 10.

Conclusion

We hope we have addressed all of the concerns of the reviewer with this rebuttal, and we thank them for their comment which will allow us to improve the paper. We have included preliminary results from the ongoing experiment and will add the final results once training is complete. We remain at the reviewer's disposal, should any point remain unclear or any new question arise.

Thanks for the response.

I believe the contribution is incremental. Yes, I acknowledge the delta between existing VAE works and this work; but it is not substantial. The training/test/val split variation is also explained vaguely. Valid molecules percentage is relatively worse for Flow model combinations.

I keep my score for the time being. I will discuss with AC and other reviewers in the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.