acceptodds
Under review as a conference paper at ICLR 2027

Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion

Abstract

Denoising graphs is a fundamental problem in graph learning and the core operation of graph diffusion models. Attention-based architectures such as graph transformers have shown promise for this task, yet the mechanisms through which attention denoises graphs remain poorly understood. We show that, under a graph denoising objective, standard attention scores learn an average spectral denoising filter across the training distribution and are suboptimal when the graph spectra vary. To address this limitation, we introduce spectral attention scores, which are attention scores that depend on the input graph spectrum. We characterize the improvement over standard attention scores in terms of the spectral diversity of the graph distribution. We then propose graph convolutional attention (GCA), a practical realization of spectral attention that uses graph-filtered queries and keys. For asymptotically large stochastic block models, we show that GCA matches the optimal denoising loss of spectral attention under suitable conditions. We also show that softmax can further reduce denoising error by approximately projecting noisy eigenvectors onto the clean eigenspace. Across synthetic and real datasets, replacing standard query–key projections with GCA improves graph denoising and diffusion, with denoising gains correlated with spectral diversity. Furthermore, replacing the DiGress architecture with a GCA-based architecture within the DiGress generative framework yields comparable generation quality with significantly faster inference.

Primary area:
generative models
Keywords:
Diffusion Models, Graph Diffusion, Generative Models, Graph Transformers, Graph denoising

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SmKROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to proove the suboptimality of graph transformers for graph denoising when compared to other operators such as graph spectral attention or the proposed graph convolutional attention (GCA) operator. Then experiment on the three frameworks to prove the theory.

Critical strengths
  • the paper is clear
  • easy to follow
  • builds on prior theory
Critical weaknesses
  • propositions 1 and 2 are not self contained and one needs to look at the appendix to see their meaning.

  • Also hard to know what is the justificaiton for using the three classes directly in the loss, as they don't guarantee recovering A even if in some cases U is recovered one still needs the eigenvalues. Moreover, only the spectral and GCA may attempt that so the standard scores are already biased from start as it has no eigenvalue info from your definitition

  • Some inovation concern is that the whole model is adding filtering to the features before the attention layers? or is there something more we are missing?

  • lines 211-213 makes S equal to A but no motivation given?

  • lines 223-224 assume the leaned weights are diagonal which usually is not?

Questions for authors

Please wee weaknesses above

1 comment

W1

We thank the reviewer for this comment. We believe the propositions are largely self-contained in the current presentation. Proposition 1 states the minimum loss achievable with standard attention scores and identifies the optimal score matrix as E[B]\mathbb{E}[B]. Proposition 2 gives the corresponding result for spectral attention scores and identifies the optimal map as h⋆(Λ~)=E[B∣Λ~]h^\star(\widetilde{\Lambda})=\mathbb{E}[B\mid\widetilde{\Lambda}]. The proofs, of course, are provided in the appendix due to space constraints. That said, we are open to improving the exposition. It would be helpful if the reviewer could clarify which definitions or aspects of the propositions they find difficult to follow, so we can address the issue more directly.

W2

We thank the reviewer for asking about the motivation for these classes. We organize the analysis in two stages. In Section 2, we study how attention scores can denoise the eigenvalues while expressed in the noisy eigenbasis. In Section 3, we build on this score analysis and study the full attention map. We show that the softmax applied to the scores can further denoise the graph by removing noise from the eigenvectors. The three classes in Section 2 differ in how their scores can use the noisy eigenvalues. Standard attention has scores of the form U~WQWK⊤U~⊤\widetilde U W_QW_K^\top\widetilde U^\top, where WQW_Q and WKW_K do not depend on the noisy eigenvalues. Spectral attention replaces WQWK⊤W_QW_K^\top with an arbitrary matrix-valued function h(Λ~)h(\widetilde\Lambda) of the noisy spectrum. GCA further restricts this dependence to a pointwise function η(Λ~)\eta(\widetilde\Lambda), applying the same scalar function to each noisy eigenvalue. This last form can be implemented using graph filters. We do not expect each class to recover AA on its own. In particular, the first class cannot in general recover the eigenvalues of AA. It serves as a reference for measuring what is gained by using the noisy eigenvalues. The spectral class describes the most general dependence on them, and GCA shows how a graph filter can provide a practical form of that dependence. Section 3 then shows how the softmax builds on these scores to remove noise from the eigenvectors, complementing the eigenvalue denoising studied in Section 2.

W3

In GCA, graph filters form the query and key representations, replacing the learned linear projections used in standard attention. Their product forms the attention-score matrix, so the filtering determines how the scores depend on the noisy graph spectrum. Furthermore, on the theoretical side, the paper uses spectral analysis to explain why this change can help graph denoising. We show that standard attention scores learn an average spectral denoising rule across the training distribution, which can be suboptimal when graph spectra vary. Allowing the scores to depend on the noisy eigenvalues can reduce the denoising loss, and GCA provides a practical way to implement this dependence. Another key aspect of the paper is our analysis of the softmax, which shows that it can further denoise the graph by approximately projecting the noisy eigenvectors onto the clean eigenspace. To the best of our knowledge, this theoretical explanation of the denoising role of softmax is novel, and no similar analysis has appeared in prior work.

W4

The adjacency matrix AA and the graph Laplacian LL are standard choices for the graph shift operator SS. We set S=AS=A because the quantity being denoised is AA itself, so its eigendecomposition gives a direct spectral representation of the target. This choice also keeps the theoretical analysis simple. We will add this motivation to the paper.

W5

We thank the reviewer for pointing out that Eq. (14) uses diagonal weights. In the theoretical construction, the input is the full eigenvector matrix UU. We impose diagonal HQ(l)H_Q^{(l)} and HK(l)H_K^{(l)} to filter each eigenmode independently. This isolates the spectral response of the filters and lets us show that polynomial graph filters can approximate pointwise functions of the eigenvalues. The diagonal restriction is used for this approximation argument. It is not an assumption that graph-convolution weights are generally diagonal. The practical GCAT model uses a more general parameterization. At each layer, it applies polynomial graph filters to the feature matrix XX with full, non-diagonal learned weights. For example, with two filter taps, Q=XWQ,0+AˉXWQ,1Q=XW_{Q,0}+\bar A XW_{Q,1}, with an analogous expression for KK. Thus, the theoretical construction shows how graph filters can implement pointwise spectral responses, while the practical model uses general graph-filtered queries and keys. We will make this distinction clear in the paper.

We would be happy to address any remaining concerns you may have. We kindly ask the reviewer to consider our responses, as well as the additional, more robust experimental results provided in the global comment.

Reviewer_UoXmOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper gives a spectral analysis of graph denoising attention and derives an architecture from it. Proposition 1 shows standard attention learns an average spectral rule; Proposition 2/Theorem 1 show the improvement of spectral attention equals exactly the spectral diversity SD(D_G); Theorem 2 shows that under an asymptotic SBM-plus-Wigner-noise model the optimal spectral rule is outlier shrinkage eta_out(lambda_tilde) = sqrt(lambda_tilde^2 - 4v^2) for |lambda_tilde| > 2v (a BBP-type phase transition), implementable by graph-filtered Q/K (GCA); Theorem 3 shows softmax at suitable temperature approximates projection onto the clean feature space, further reducing basis-mismatch error. Experiments: GT vs GCAT denoising on 5 unattributed datasets (validation loss reduced 20-33%, correlated with estimated spectral diversity at r = 0.81); an equal-capacity GCAT-256 vs DiGress generation comparison (comparable quality, 1.7-8.3x faster per inference step, scaling n^0.39 vs n^0.89).

Critical strengths
  1. The theoretical chain is complete and self-consistent, and the spiked-Wigner asymptotics are used elegantly to derive a concrete, implementable spectral rule.

  2. The measurement methodology is unusually careful: a permutation-null-calibrated spectral-diversity estimator, dual-reference normalized MMD, honest N/A markings, and a compute ledger.

  3. Theory and experiments genuinely interact - the predicted correlation between spectral diversity and denoising gain is actually tested rather than assumed.

Critical weaknesses
  1. The flagship denoising comparison lacks the decisive control: GT has 25,796 parameters vs. GCAT's 42,692 (66% more), and GCAT additionally consumes top-16 eigenvector inputs that GT does not receive. All 20-33% gains and the reported correlation come from this unequal pairing, while the equal-capacity control was only run for the generation arm. By the paper's own ledger, each denoising run costs only 7-11 minutes, so the control is cheap to add - its absence is difficult to excuse.

  2. The theory-practice bridge is statistically thin: r = 0.81 is computed over only n = 5 points (p ~ 0.10, not significant; after calibration r ~ 0.71), and one of the five points is a known outlier. Moreover, the theorem regimes are far from the experiments: Theorem 2 assumes Wigner noise, while the edge-flip noise used empirically induces a (1 - 2eps) attenuation and an eps*11^T rank-one spike that are never analyzed, and Theorem 3 requires p << min(1/n^2, 1/(k^2 log^2 k)), several orders of magnitude below the experimental eps = 0.05-0.2.

Questions for authors

(1) (Decisive) Under an equal-capacity, equal-spectral-input GT control, do the 20-33% denoising gains and the correlation with spectral diversity survive?

(2) Why do GT/GCAT have no time-step conditioning? Cross-family loss comparisons then mix two handicaps; what happens when conditioning is added?

(3) What are the leave-one-out interval and p-value for r, and what does the calibrated r ~ 0.71 imply for the abstract's claim?

(4) For the experimental graphs (e.g., ENZYMES at eps = 0.2), what are the actual values of E||E||_2, m, k, F - do Theorem 3's conditions hold at all?

(5) For class-flip noise, how do the (1 - 2eps) attenuation and the eps*11^T spike affect eta_out and the optimality claim?

(6) The abstract's "improves denoising and diffusion" conflicts with Fig 3b (DiGress is better on the ENZYMES/DEEZER/COLLAB loss rows and the DEEZER MMD), and "significantly faster" is reported without any significance test; the conclusion's "simpler" conflicts with GCAT having 66% more parameters.

(7) The generation comparison uses only 1-2 seeds and 12-24 samples.

(8) Absolute s_norm values are not reported, and the per-step top-k eigendecomposition cost is not counted in the scalability claim.

2 comments

W1

We thank the reviewer for highlighting this point. First we clarify that GT and GCAT both receive the top 16 eigenvectors as their input tokens (Line 349). We reran the denoising and generation experiments for capacity matched GCAT (24,340 params) and GT with LayerNorm (26,308 params). This isolates the convolutional attention mechanism as the only point of difference between the two models. In Table 6a of the global response we provide the results across the five original datasets plus three new datasets from TUdatasets, with three training seeds for each dataset-model pairing. Furthermore, we increased the number of generated samples used for the MMD evals to 120 graphs for each model-dataset pair. The results show the same trends as the heatmap in Figure 3-a of the original submission. Validation denoising loss is improved significantly across the board when using GCAT vs GT. The generative MMD metrics also favor GCAT across most cells but the improvement is not as uniform as the denoising loss.

W2

"...r=0.81r = 0.81 is computed over only n=5n = 5 points"

This is a good point. We would like to clarify that our goal in providing the results in Figure 2a, is not necessarily to test the statistical significance of a hypothesis, but to rather emphasize that GCAT always improves on validation loss compared to GT (as predicted by our theory) and that there does seem to be a correlation between the estimated spectral diversity and the improvement in validation loss which is in line with the trend suggested by Theorem 1. That being said, we added three new datasets from TUDataset benchamarks to our comparisons, and they also follow the expected trend. With the 8 datasets in total correlation not only increases slightly (r = 0.84) but it also becomes statistically more significant (p = 0.01).

(See Figure 5a in revised pdf/Table 5a in the global response.)

"Theorem 2 assumes Wigner noise..."

The reviewer raises a very relevant point. We use Wigner noise in our theoretical analysis because it is a well-studied model that allows us to derive results and gain intuition. In practice, graph diffusion models use different noise models. Some methods use Gaussian noise in a continuous representation, while edge-flip noise is also commonly used for discrete graphs, as in our experiments. As shown in Appendix D.8, the edge-flip model can be decomposed as

A~=A+E,E=−2εA+ε11⊤+Ξ,\widetilde A=A+E,\qquad E=-2\varepsilon A+\varepsilon\mathbf 1\mathbf 1^\top+\Xi,

where Ξ\Xi is a centered Wigner-type matrix. Equivalently, we can write

A~=A^+Ξ,A^=(1−2ε)A+ε11⊤.\widetilde A=\widehat A+\Xi, \qquad \widehat A=(1-2\varepsilon)A+\varepsilon\mathbf 1\mathbf 1^\top.

If the constant vector is an eigenvector of AA, as it is for many graph families, including SBMs, then A^\widehat A has the same eigenbasis as AA. The nonconstant eigenvalues are scaled by 1−2ε1-2\varepsilon, while the eigenvalue associated with the constant vector receives an additional ε\varepsilon-dependent shift. We can therefore apply Theorem 2 to A^\widehat A with only minor changes. The shrinkage function ηout\eta_{\mathrm{out}} is evaluated at the eigenvalues of A^\widehat A, which depend on the original eigenvalues and on ε\varepsilon.

"Theorem 3 requires p≪min⁡(1/n2,1/(k2log⁡2k))p \ll \min(1/n^2, 1/(k^2 \log^2 k)), several orders of magnitude below the experimental ϵ\epsilon"

The reviewer is correct that the assumption for Theorem 3, requires the noise level to be significantly smaller than what we use in our experiments. However, in our view, the significance of Theorem 3 lies in the fact that it is (to the best of our knowledge) the first result of its kind which analyses the role of the softmax nonlinearity on the reconstruction of the graph from a spectral perspective. This helps us give a full theoretical picture of the denoising mechanisms of the transformer from a spectral view. Our experimental results show that the use of the softmax improves denoising well beyond the small noise regime required by the theorem. This suggests to us that with more extensive analysis theorem 3 could potentially be expanded to a regime with larger noise which is a direction we are exploring for future work.

We respond to the reviewer's questions in the following official comment.

Q1

Yes, the gains and the correlation both remain strong in the equal-capacity case. We note that the results in the paper are already with equal-spectral-input. See our response to weakness 1 above and Tables 5a and 6a in the global response.

Q2

We note that since DiGress does have time-step conditioning, this handicaps GCAT compared to DiGress which suggests that there might be room for even further improving the performance of GCAT compared to Digress. Adding a simple time step conditioning mechanism to GCAT we observed that its validation edge loss slightly improved on average across datasets by 4.8%.

Table A1. Timestep-conditioning ablation: relative reduction in validation edge cross-entropy from adding it to GCAT (662k → 697k parameters).

ENZYMESPROTEINSIMDB-BCOLLABDEEZERIMDB-MREDDIT-TDHFR
Edge CE (rel.)+6%+5%+3%+6%+7%−3%+8%+7%

Q3

With adding three more datasets to our comparisons we see an improved correlation and p-value which strengthens our claim in the abstract. Leaving out any one of the eight datasets, the correlation stays between r=0.81r = 0.81 and r=0.91r = 0.91, with p≤0.028p \le 0.028 in every case. See our response to weakness 2 above for details.

Q4

We do concur that theorem 3's assumptions do not quite hold in practice but that the experimental results show that the improvement of using the softmax which the theorem explains extends well beyond the small noise regime. See our response to weakness 2 above.

Q5

Theorem 2 holds for edge flip noise as well with minor modifications. See our response to weakness 2 above.

Q6

We clarify that "improves denoising and diffusion" was meant with respect to the standard graph transformer (GT). Regarding speed, in the revised results (Figure 5b and Table 2 in the appendix page 15) we observe that GCAT samples between 3.0x to 7.2x times faster in wall clock time than digress across the different datasets (1.5x to 3.6x per reverse diffusion step) which we believe supports our claim of 'significantly faster'. If the reviewer is looking for other specific metrics, we would be happy to provide them. Also regarding the 'simpler' claim in the conclusion, we note that we claim GCA is simpler than DiGress, and that the simplicity is meant not in terms of parameter count, but rather the architecture itself. DiGress has significant modifications on top of a graph transformer (such cross modulation with FiLM layers and global feature streams). We see that GCAT shows competitive performance to Digress while being simpler and much faster at inference time. That said, if the reviewer disagrees with any of these explanations, we would be happy to revise the abstract per their suggestion.

Q7

To further verify the robustness of the results, For the revised equal capacity comparisons we have used three seeds and 120 generated graphs for generative evaluation (MMD values) per model-dataset pairing.

(See Tables 6a and 6b in the global response.)

Q8

"Absolute snorms_{\mathrm{norm}} values are not reported"

We have added them to the revised paper in Table 1 of the Appendix (page 15) that reports the absolute snorms_{\mathrm{norm}} of every model, for each dataset and metric.

"the per-step top-kk eigendecomposition cost is not counted in the scalability claim"

We clarify that the reported times do include the eigendecomposition. It is computed inside the model's forward pass at every reverse step, and we measure the end-to-end sampling time. Both GCAT and DiGress use the eigenvector inputs in this comparison, so both models pay this cost. With the greater number of seeds and the additional datasets, now the average wall clock time improvement of GCAT vs DiGress ranges between 3.0x and 7.2x (1.5x to 3.6x per reverse diffusion step).

We would be happy to address any remaining concerns the reviewer may have, particularly those related to question 1, which was marked as decisive.

Reviewer_gC1qOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Graph Convolutional Attention (GCA), using graph-filtered queries and keys to improve graph denoising and diffusion. Spectral analysis motivates the method, and experiments report improved denoising and faster inference than the evaluated baselines.

Critical strengths

(1) The paper connects spectral variability to denoising performance and separates the roles of eigenvalue adaptation and softmax-based eigenspace denoising, providing an interpretable motivation for GCA. (2) GCA replaces standard query–key projections with learnable graph filters, making the proposed mechanism straightforward to incorporate into transformer architectures. (3) GCAT reduces validation loss by 20–33% relative to GT across five datasets. At matched capacity, it also achieves 1.7–8.3× faster inference per diffusion step than the tested DiGress variant, with mixed but broadly comparable generation quality.

Critical weaknesses

(1). The central theoretical result overlooks the attention rank constraint. In Eq. (4), WQWK⊤W_QW_K^\top has rank at most dkd_k, but Proposition 1 identifies the optimum as E[B]\mathbb{E}[B], which need not satisfy this constraint. Theorem 1 therefore requires an additional rank assumption or a rank-approximation term for its claimed equality with spectral diversity to hold.

(2) The experiments do not sufficiently isolate GCA’s contribution or establish generation robustness. The GT–GCAT comparison increases parameters from 25,796 to 42,692 and adds query/key LayerNorm, so the gains cannot be attributed solely to graph filtering. Generation evaluation also uses only 24 generated graphs per run, with the GCAT–DiGress heatmap based on one training seed, limiting confidence in the reported quality comparisons.

Questions for authors

Appendix B.7 mentions filter-tap and normalization ablations that were conducted but not reported. What do these results show about the contribution of graph filtering, and do any existing controls separate its effect from the additional parameters and query/key LayerNorm?

1 comment

Response to Reviewer gC1q

W1

We thank the reviewer for identifying this omission. The reviewer is right that Eq. (4) imposes a rank constraint, since rank⁡(WQWK⊤)≤dk\operatorname{rank}(W_QW_K^\top)\leq d_k, whereas the proof of Proposition 1 minimizes over an unconstrained matrix WW. Under the rank constraint, the best estimator is the rank-dkd_k truncated SVD of M=E[B]M=\mathbb{E}[B], by the Eckart–Young theorem. Let MdkM_{d_k} denote this approximation and define

εdk=∥M−Mdk∥F2=∑i>dkσi(M)2.\varepsilon_{d_k} = \lVert M-M_{d_k} \rVert_F^2 = \sum_{i>d_k}\sigma_i(M)^2.

The minimum standard-attention loss is then

=\operatorname{tr}\left(\operatorname{Cov}(\operatorname{vec}(B))\right) +c+\varepsilon_{d_k},$$ and the comparison with spectral attention becomes $$\ell_{A,d_k}^\star-\ell_{SA}^\star =\mathrm{SD}(D_G)+\varepsilon_{d_k}.$$ Thus, the rank constraint does not change the qualitative conclusion that spectral attention improves on standard attention. It adds a nonnegative term to the improvement, while spectral diversity remains one of its components. The equality in Theorem 1 as currently stated holds when $\operatorname{rank}(\mathbb{E}[B])\leq d_k$. Since the main purpose of Theorem 1 is to isolate the role of spectral diversity, we believe the cleanest presentation is to state this rank assumption explicitly. If the reviewer considers it important to avoid this assumption, we can instead present the fully general rank-constrained result, including the additional term $\varepsilon_{d_k}$ due to the low-rank approximation. Once we hear back from the reviewer, we will upload the manuscript accordingly. ## W2 > "The experiments do not sufficiently isolate GCA's contribution" We thank the reviewer for pointing out this issue. We isolate GCA's contribution by comparing GCAT at nearly the same parameter count as GT (24.3k parameters for GCAT vs 26.3k parameters for GT) while also adding LayerNorm to GT. We see that GCAT still outperforms GT uniformly across all 8 datasets (the original 5 from the submission + 3 more datasets from TUDatasets added for the rebuttal) in terms of validation loss, and also improves on GT on most of the generation metrics. Please refer to Table 6a in the global response/Figure 6a in the revised pdf for the full results. > "or establish generation robustness" To better establish generation robustness, in the new capacity matched comparison, we have used three training seeds and 120 generated graphs for generative evaluation (MMD values) per model-dataset pairing. We have also redone the Digress vs GCAT comparisons with the same revised setup (three training seeds and 120 generations for eval). Similar to the results presented in the paper we observe that in some datasets GCAT does slightly better and in some DiGress does slightly better in terms of val loss. However for generative metrics, GCAT often has an edge. Please refer to Table 6b in the global response/Figure 6b in the revised pdf for the full results. ## Q1 As mentioned in our response to Weakness 2 above, we isolate the contribution of graph filtering in the attention by matching the number of parameters across GT and GCAT, and adding query/key LayerNorm to GT. We believe the fact that GCAT retains a strong advantage in this otherwise matched comparison supports the effectiveness of the GCA mechanism (Table 6a). In our ablations on number of filter taps $p$, increasing it beyond $p=2$ makes performance slightly better on some datasets and slightly worse on others so we chose $p=2$ for the main experiments in the paper. We would be happy to address any remaining concerns that the reviewer may have.

Comments on the paper

2 comments

Global Response

We thank the reviewers for their valuable feedback. The main experimental concerns were that the GT/GCAT comparison was not capacity-matched, that the correlation with spectral diversity rested on five datasets, and that generation was evaluated with few seeds and samples. We provide experimental results to address all three:

  • Capacity-matched GT vs GCAT (Tables 5a, 6a). GCAT (24.3k parameters) against GT with Q/K LayerNorm (26.3k), so convolutional attention is isolated as the only difference. The results are in line with the original submission. GCAT lowers validation loss on all 8 datasets (10–28%) and is better across most generative MMD metrics.
  • Three new datasets (Table 5a). Adding IMDB-MULTI, REDDIT-THREADS and DHFR, further showcases robustness of the results and makes the correlation between spectral diversity and improvement statistically more significant: Now we have Pearson correlation r=0.84r = 0.84 (p=0.010p = 0.010) over 8 datasets.
  • More robust generation evaluation (Tables 6a, 6b). All comparisons now use three training seeds and 120 generated graphs per checkpoint.

We present the results in figures added in page 10 of the edited paper pdf. We kindly ask the reviewers to take a look at the new figures in page 10. For the reviewers' convenience, we provide the results as tables here. In all tables, means are over 3 training seeds. MMD cells are (baseline −- GCAT) divided by (ER −- real–real); positive (bold) means GCAT is better; n/a where the ER reference does not separate from real–real. (similar to the paper)

Table 5a. (Matching Figure 5a in the revised paper) Spectral diversity g^\hat g (Appendix C.6; ±\pm sd over three estimation seeds) and improvement in final validation edge cross-entropy, GT −- GCAT (±\pm sd over seeds). Pearson correlation r=+0.84r = +0.84 (p=0.010p = 0.010).

Datasetg^×102\hat g \times 10^{2}improvement (GT −- GCAT)
DHFR0.20±0.020.20 \pm 0.020.039±0.0070.039 \pm 0.007
REDDIT-THREADS0.57±0.040.57 \pm 0.040.026±0.0020.026 \pm 0.002
DEEZER-EGO-NETS0.63±0.020.63 \pm 0.020.053±0.0130.053 \pm 0.013
ENZYMES1.23±0.051.23 \pm 0.050.093±0.0050.093 \pm 0.005
PROTEINS1.64±0.141.64 \pm 0.140.049±0.0070.049 \pm 0.007
IMDB-MULTI2.54±0.072.54 \pm 0.070.163±0.0540.163 \pm 0.054
IMDB-BINARY2.94±0.172.94 \pm 0.170.171±0.0440.171 \pm 0.044
COLLAB4.83±0.254.83 \pm 0.250.154±0.0110.154 \pm 0.011

Table 6a. (Matching Figure 6a in the revised paper) GCAT (24.3k parameters) against GT with Q/K LayerNorm (26.3k parameters). Last row: relative reduction in validation loss, (GT −- GCAT)/GT.

ENZYMESPROTEINSIMDB-BCOLLABDEEZERIMDB-MREDDIT-TDHFR
Degree MMD+0.64+0.61+1.77+0.06−0.15n/a−0.12−0.01
Clustering MMD+0.64+0.61+1.08+0.40−0.13+6.40+0.74+0.07
Spectral MMD+0.45+0.23+0.46+0.21−0.01+1.57−0.09+0.25
Orbit MMD+0.78+0.19−0.20−0.15+0.02+0.91−0.12+0.08
Val. loss (rel.)+23%+18%+26%+20%+15%+28%+10%+17%

Table 6b. (Matching Figure 6b in the revised paper) GCAT (timestep conditioning, 704k parameters) against DiGress (678k). Last row: relative reduction in edge cross-entropy, (DiGress −- GCAT)/DiGress, the one loss term both models define identically.

ENZYMESPROTEINSIMDB-BCOLLABDEEZERIMDB-MREDDIT-TDHFR
Degree MMD−0.02+0.26+1.48+4.73−0.08n/a+0.02+0.16
Clustering MMD+0.33+0.05+0.92+1.16−0.01+1.72+0.03+0.36
Spectral MMD+0.18−0.33+0.53+1.30+0.00+0.53+0.02−0.07
Orbit MMD+0.62+0.22+0.35+2.20−0.01+0.34−0.03−0.22
Val. loss (rel.)−2%+12%−2%+4%−0%+5%−2%−3%

Thank you to the reviewer who has already followed up on the authors' responses. For reviewers who have not yet done so, please post a brief follow-up acknowledging the authors' responses within the next 24 hours, so the authors have time to respond before discussion closes. In your reply, please indicate which main concerns have been addressed, what remains, and whether your overall assessment changes or stays the same.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.