- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
- In the context of molecular GNNs, dense attention fails to provide physically meaningful attribution for feature binding (attention tends to pick up statistical artifacts). This paper aims to address this problem.
- Proposal: MolSync, a molecular GNN based on nonlinear phase synchronization dynamics (a vector Kuramoto model on the sphere).
- It learns molecular representations through feature binding (self-organization into functional substructures), and claims that continuous synchronization removes the multi-hop (finite-hop) constraint of fixed-depth MPNNs.
- The core part of the model is imported from AKOrN (Miyato+, image domain) to molecular graphs. Architecture: multi-scale initialization via Geometric Scattering Transform (GST) -> driven Kuramoto updates (L blocks x T internal steps) -> phase-invariant readout. Energy-based attribution (EBA) is also provided as a sub-product.
- Experimentally validated with the MoleculeNet test sets.
Critical strengths
- DeMol and Uni-Mol require 3D conformer information. Compared with these, MolSync achieves competitive scores on MoleculeNet using only 2D graph information.
- Introducing the Kuramoto model into molecular GNNs as an alternative inter-node interaction mechanism to attention is new to me and interesting.
Critical weaknesses
- Experiments are not strong.
- The claim in Table 1 ("best on 6/8 datasets") sounds overstated to me. The margin over the strongest baseline DeMol is very thin. MolSync vs DeMol: BBBP +3.2, but Tox21 +0.5, SIDER +0.2, ClinTox +0.3, MUV +0.1, BACE +0.1. It seems these margins are not significant compared to the reported std (0.2–1.4) except for BBBP.
- Ablations (Table 2) and qualitative (Figure 3) evaluation only on BBBP, where the effect is largest. Its generality is unclear since no validation on other datasets is provided.
- Figure 3 lacks quantification. Without a comparison against a globally collapsed picture, or a quantitative metric, it is difficult for readers to tell whether the figure is consistent with Thm. 2. For example, do attention-based models exhibit a similar grouping structure or not?
- In Table 2, the largest drops are w/o Kuramoto (73.8) and w/o GST (73.9), essentially tied. This means initialization (GST) matters a lot. However, the computational cost of GST (preprocessing) is not reported in the complexity analysis (Table 5).
- Rationale for the model selection, and induced theory
- If my understanding is correct, the update rule is close to the original (scalar) Kuramoto model, and may be weaker than AKOrN (Miyato et al.) in terms of representation capacity. Is reproducing the Kuramoto form itself essential? Would an implementation faithful to AKOrN perform poorly?
- Separation between global phase collapse and over-smoothing: Thm. 2 only shows that exact global collapse is impossible. Approximate homogenization of representations (over-smoothing in practice) is not ruled out, and how these are separated is unclear.
Questions for authors
- Do you have any significance test results for the performance differences in Table 1?
- What is the total number of trainable parameters? What is the computational cost of GST preprocessing?
- Is the grouping structure in Figure 3 also observed for attention-based models? A relative comparison would be appreciated.
- What is the essential implementation difference from AKOrN? How does performance compare against an implementation faithful to AKOrN?
- What is the essential reason the dynamics must be Kuramoto-type?
- Around L325: how do you distinguish the global phase collapse from the over-smoothing? How does "exact collapse does not occur" relate to avoiding over-smoothing?
11 comments
Rebuttal Part2
W2 — Ablations and qualitative evaluation only on BBBP
We accept the concern about relying on a single-dataset ablation. We have extended the component ablations to all eight classification datasets, using five independent runs per configuration under the same evaluation protocol.
| Variant | BBBP | BACE | ClinTox | SIDER | HIV | Tox21 | MUV | ToxCast | Avg |
|---|---|---|---|---|---|---|---|---|---|
| w/o GST | 73.9 | 87.1 | 92.6 | 63.7 | 79.5 | 79.4 | 77.9 | 65.6 | 77.46 |
| w/o Kuramoto | 73.8 | 86.8 | 88.5 | 65.8 | 78.9 | 76.8 | 77.9 | 65.8 | 76.79 |
| w/o phase readout | 76.8 | 87.4 | 90.2 | 67.8 | 79.3 | 80.1 | 82.0 | 67.1 | 78.84 |
| MolSync (full) | 78.3 | 89.1 | 92.9 | 68.6 | 80.9 | 81.4 | 82.6 | 67.8 | 80.20 |
On the eight-dataset average, removing Kuramoto evolution reduces ROC-AUC by 3.41 points, removing GST by 2.74 points, and removing the phase-invariant readout by 1.36 points. Each removal decreases the reported mean on every dataset. Thus, the components contribute beyond BBBP, with the largest average drop observed when the dynamical evolution is removed. These are component ablations within MolSync; the independent value of its overall architecture is further assessed with GST-equipped GIN controls in W4/Q2. These ablations are reported as Table 7 (Appendix D.5) of the revised paper.
The group-structure analysis in W3/Q3 remains an analysis of BBBP; we do not use that result to claim the same structure has been quantitatively established on every dataset.
Rebuttal Part3
W3 & Q3 — Quantifying Figure 3, and comparison with attention models
What Figure 3 measures. Figure 3 is not an attention map. For the final node states before pooling, we first subtract the molecular mean, normalize each nonzero centered vector, and calculate the mean cosine similarity between atoms belonging to structural groups and . The same representation-based measurement can be applied to MolSync and a Transformer, rather than comparing unrelated internal attention weights.
Correcting self-pair inflation. The submitted diagonal included self-pairs, each with cosine similarity 1. For a group of size , its reported diagonal satisfies
where averages over distinct within-group atoms. We now use and exclude singleton groups; otherwise, small groups acquire artificially high diagonal values.
Controlled comparison. Using the same BRICS group partition and the same metric, we compare MolSync with a Transformer on the 93 BBBP molecules both models classify correctly (471 non-singleton groups). This common-correct subset controls for prediction outcome but is not claimed to represent the entire test distribution.
| Metric | MolSync | Transformer |
|---|---|---|
| Groups with | 334/471 (70.9%) | 227/471 (48.2%) |
| Per-molecule dominance rate (mean) | 0.769 | 0.548 |
| Binding contrast | 0.530 ± 0.185 | 0.439 ± 0.208 |
| Within-group cosine | 0.427 | 0.376 |
| Between-group cosine | −0.103 | −0.063 |
We use the molecule, rather than each of its groups, as the inferential unit. MolSync has a higher per-molecule dominance rate in 62/93 molecules (paired Wilcoxon , as reported from the molecule-level analysis). The paired binding-contrast difference is +0.091 (95% CI [+0.039, +0.144], paired Wilcoxon ). We do not treat all 471 groups as independent observations in a chi-squared test, nor do we describe the Transformer's 48.2% as chance level without an appropriate group-size-aware null distribution.
The negative average between-group similarity indicates relative separation in the centered representation space. Because the representations are centered before normalization, it should not by itself be interpreted as proof of active repulsive synchronization, nor is a well-defined diagnostic of exact global collapse (centered vectors would approach zero). Our empirical conclusion is narrower and directly supported: MolSync exhibits stronger within-versus-between group organization under the same measurement than this Transformer baseline. These statistics are reported as Table 8 (Appendix D.6) of the revised paper.
Selection robustness. Figure 3 presents ten molecules selected for visual clarity. Removing those illustrations leaves dominance rates of 287/419 (68.5%) for MolSync and 201/419 (48.0%) for the Transformer, close to the full-sample comparison. We will describe Figure 3 as illustrative rather than as a randomly sampled average case. The ten molecule-level comparisons are reproduced as Figure 7 (Appendix D.6) of the revised paper.
Rebuttal Part1
Response to Reviewer GwWf
We thank you for a detailed and technically precise review, and for recognising the synchronisation idea and the substructure connection as strengths. Four of your concerns led to changes that materially improved the paper: the ablation now covers all eight datasets, Figure 3 is quantified against an attention baseline, the computational cost of the GST component is measured and reported, and the discussion of Theorem 2 has been rewritten to make the role of the Dirichlet energy bound explicit. We have also added a finite-step separation result that speaks to Q5. We address every point below.
W1 & Q1 — Margins in Table 1, and significance testing
You are right that the margins were reported without a test. We have now run one on all eight benchmarks and we report it in full, including the dataset on which the test goes against us.
Test. For each benchmark we compare our five independent MolSync runs against the strongest published mean in Table 1 using Welch's unequal-variance t-test computed from the reported summary statistics (mean, standard deviation, run count), two-sided, with Holm--Bonferroni correction across the eight datasets.
Outcome: one separation in our favour (BBBP), one against us (ToxCast), and six benchmarks on which MolSync and the strongest pre-trained 3D baseline are statistically indistinguishable. Note that this is the correct reading of HIV as well: the −0.3 deficit against DeMol is within run-to-run variation and is not a separation in either direction.
Robustness to the one unknown. The compilation does not state the run count behind the baseline rows. We therefore repeated every test under both nbase=3 and nbase=5. The verdicts are identical in all eight cases, and the two significant results move only in the third decimal (p=0.017 and 2.3×10−5 at nbase=3). The conclusion does not depend on that assumption.
What the test licenses. Because baseline per-seed values are not published, this is an unpaired comparison against summary statistics rather than a paired test on common splits, and it inherits whatever protocol differences exist between the cited studies and ours. We state this limitation with the numbers. Where both arms run inside our own pipeline on identical splits, paired testing is available and we use it: the GST-equipped GIN controls in W4 & Q2 and the faithful AKOrN comparison in W2.1 & Q4, each over five runs per arm.
How Table 1 is now described. The average column has been recomputed directly from the eight displayed entries for every row (DeMol 79.88, MolSync 80.20), and the surrounding text states the result the test supports: MolSync is statistically indistinguishable from large pre-trained 3D-conformer models on six of eight MoleculeNet classification benchmarks, higher on BBBP by 3.2 points, and lower on ToxCast, while being trained from scratch on each benchmark's training split from the 2D bond graph alone. We make no claim of a benchmark sweep, and the per-dataset verdicts above are reported as they came out.
The reason we still consider this worth reporting is the asymmetry in what each side consumes: GEM, Uni-Mol and DeMol are pre-trained on millions of unlabelled molecules and take 3D conformers as input. Reaching parity with them from 2D topology and no task-independent pre-training is the claim the architecture is meant to support, and it is the claim the test leaves standing.