acceptodds
Under review as a conference paper at ICLR 2027

Distance is More Than a Bias: Geometry-Driven Value Mixing in Graphs

Abstract

Geometric deep learning and graph transformers have driven significant advancements in learning from structured data, particularly in molecular property prediction and spatial modeling. However, while recent approaches often incorporate spatial distances as attention biases or complex pair representations, this significantly increases computational overhead. In this paper, we propose a paradigm shift: rather than treating distance merely as a bias, we utilize the distance matrix as a direct value-mixing operator to achieve both high predictive performance and extreme computational efficiency. We introduce DisAttention, which replaces standard query-key attention with mixing weights derived from learnable transformations of the distance matrix. To further enhance expressive power, we propose DisMix, which integrates node feature affinity through symmetric normalization and distance gating instead of conventional row-wise softmax. By eliminating the need to update high-dimensional pair states, our geometry-driven approach drastically simplifies computation. Extensive experiments across molecular datasets (QM9, MolPCBA, LP-PDBbind) and broader geometric domains (MNIST superpixels, ModelNet40) demonstrate that our models achieve highly competitive accuracy while significantly reducing parameter count, peak memory usage, and training time compared to state-of-the-art graph transformers.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Transformers, Geometric Deep Learning, Efficient Graph Neural Networks, Molecular Property Prediction, Continuous Distance Filter

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Fj1DOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes using the pairwise distance matrix directly as the value-mixing operator in graph transformers and introduces DisAttention. It derives mixing weights from distances only, and DisMix adds query-key affinity, replacing softmax with normalization and a learned distance gate. Experiments on molecular and geometric benchmarks show competitive accuracy with fewer parameters, lower memory, and faster training.

Critical strengths
  • The paper proposes a new way to incorporate distance embeddings in graph transformer architectures
  • The method makes the model and training faster and more efficient
Critical weaknesses
  • Lack of comaprisons: Distance-based message weighting is not new. EGNN (https://arxiv.org/pdf/2102.09844) uses pairwise distances to modulate messages, and SchNet uses continuous filters on interatomic distances. Neither is compared on QM9. More broadly, the QM9 baselines are limited and omit standard geometric models such as DimeNet, PaiNN, or equivariant transformers. Without these, it is hard to assess where the method stands.
  • Relation to positional encodings: Graph transformers already incorporate structure through positional encodings, including relative, spatial, and Laplacian-based variants. These are also lightweight ways to inject positional dependence. The paper should clarify how distance-driven mixing differs from these approaches, ideally with a direct empirical comparison.
  • The proposed models do not achieve the best test AP on MolPCBA, and several baselines perform better. Parameter efficiency is a useful property, but it is not compelling on its own if accuracy lags behind.
  • Expressivity studies: The paper provides no theoretical analysis of the expressive power of distance-driven mixing relative to standard attention. The empirical results also do not show a clear advantage.
Questions for authors
  • How were the coordinates for molecules in MolPCBA obtained, or is the distance matrix being computed in any other way?
  • Are node positions also input to the model?
  • Have the authors considered non-Euclidean distance measures, such as shortest-path distances on 2D molecular graphs or on graphs without spatial embeddings (e.g., social or citation networks)?
  • Is there a reason why specific function representations are chosen for ρ\rho and gg. Have the authors explored different parameterizations?
3 comments

Response to Reviewer Fj1D (1/2)

We thank the reviewer for the positive view of how distance is incorporated into the graph transformer and of the training efficiency. We also appreciate the specific comments on the comparisons and on expressivity; we added all the requested comparisons and analyses.

W1. Comparison with SchNet, EGNN, and standard geometric models

We evaluated the requested models on QM9 with the same split and training budget.

  • Against SchNet and EGNN, the distance-based models raised by the reviewer, DisMix has lower error. QM9 gap MAE↓ (300 epochs, n=3n=3, meV): SchNet 51.85±0.5651.85 \pm 0.56; EGNN 44.64±1.6244.64 \pm 1.62; DisMix 41.05±0.3641.05 \pm 0.36.
  • Against models that keep directional information, DisMix has higher error: PaiNN 40.57±0.0840.57 \pm 0.08; DimeNet++ 35.24±0.2235.24 \pm 0.22; TorchMD-Net ET 34.90±0.3134.90 \pm 0.31.
  • Our focus is to construct the continuous filter of these models as the global value mixing of a graph transformer. The global connection carries interactions beyond a cutoff. DES370K dimer interaction energies, charged dimers separated by more than 5 Å: PaiNN / DimeNet++ with a 5 Å cutoff, test MAE↓ = 1.291.29 / 2.082.08 kcal/mol; DisAttention, 0.190.19 kcal/mol (n=3n=3).

W3. Accuracy on MolPCBA

The models in the literature table (Table 1) differ in pretraining and training conditions, so we added a comparison under unified conditions; we agree that parameter efficiency matters only when accuracy is kept. Trained from scratch with the same backbone, inputs, and training budget, DisAttention obtains the highest mean AP with the fewest parameters.

  • DisAttention / Vanilla QK + distance bias: test AP↑ = 29.52±0.1329.52 \pm 0.13 / 29.22±0.18%29.22 \pm 0.18\%; parameters↓ = 9.479.47 / 12.9612.96M; training step↓ = 22.9722.97 / 28.3028.30 ms (n=3n=3).
  • DisMix / TGT-Ag: test AP↑ = 29.27±0.2229.27 \pm 0.22 / 28.75±0.16%28.75 \pm 0.16\%; training step↓ = 28.7628.76 / 44.8544.85 ms; peak memory↓ = 2.22.2 / 5.25.2 GB.

W4. Theoretical analysis of expressivity and empirical advantage

We added a theory section. The operator decomposes exactly as W=diag(m) ΠW = \mathrm{diag}(m)\,\Pi, where Π\Pi holds the relative transmission proportions and mim_i is the total transmission of node ii. When all node features are identical, the output of row-softmax attention does not depend on distances, whereas the output of our operator reflects geometry through mim_i. In addition, ∥W∥2≤2\lVert W \rVert_2 \le 2 bounds the amplification independently of graph size.

  • ModelNet40 with constant node features (n=3n=3), row-softmax / row-softmax + distance gate / DisAttention / DisMix: test accuracy↑ = 4.05±0.004.05 \pm 0.00 / 75.78±0.2175.78 \pm 0.21 / 76.97±0.5976.97 \pm 0.59 / 76.28±0.37%76.28 \pm 0.37\%.

W2. Relation to positional encodings

Positional encodings provide structural information as an input from which attention learns its weights; our work constructs the transmission weights themselves from distances. The two can be used together. For relative distance bias, the most direct counterpart, we changed only the normalization and the gate under the same distance information and logits.

  • QM9 gap MAE↓, row-softmax / row-softmax + gate / DisMix: 50.56±0.2150.56 \pm 0.21 / 49.35±0.1749.35 \pm 0.17 / 47.83±0.5347.83 \pm 0.53 meV (n=3n=3).

Q1, Q2. MolPCBA coordinates and node positions as input. We generate one conformer per molecule from SMILES with RDKit ETKDGv3, optimize it with MMFF94, and compute Euclidean distances from its coordinates. Coordinates are not node features on MolPCBA, QM9, and LP-PDBbind; ModelNet40 uses XYZ and MNIST superpixel uses the normalized 2D position as node features.

Q3. Non-Euclidean distances. The same operator can be applied to shortest-path distances, and the gain from continuous geometry depends on the task; transductive graphs without coordinates, such as social or citation networks, are not in our evaluation scope.

  • DisAttention on MolPCBA, 3D / SPD: test AP↑ = 29.35±0.2029.35 \pm 0.20 / 28.81±0.12%28.81 \pm 0.12\%; DisMix on QM9 gap, 3D → SPD: MAE↓ = 56.19±0.2256.19 \pm 0.22 → 91.22±0.6991.22 \pm 0.69 meV.

Q4. Choice of function representations and other parameterizations. The affinity G(d) eb(d)G(d)\,e^{b(d)}, a Gaussian with a learnable correction, learns the relative weights over distances. Symmetric normalization is invariant to the overall scale of the affinity, so the transmission strength is learned separately by the gate after normalization, g(d)=2σ(q(d))g(d) = 2\sigma(q(d)).

  • QM9 gap MAE↓, head-wise constant gate / gate before normalization / distance gate after normalization: 57.62±0.1857.62 \pm 0.18 / 57.90±0.3057.90 \pm 0.30 / 56.19±0.2256.19 \pm 0.22 meV (n=3n=3).

In summary, the comparison with the requested geometric models, the MolPCBA comparison under unified conditions, and the theory and controlled experiment on expressivity clarify where the method stands and under which conditions its advantage appears. Conditions, further results, and the list of revisions follow in the next comment, and we are happy to answer any further questions.

Response to Reviewer Fj1D (2/2): conditions, further results, and revisions

W1.

  • QM9 comparison: train / validation / test = 110,000 / 10,000 / 10,831, the same target normalization; each model keeps its own representation and readout and uses its published size (parameter counts are in the appendix table of Comparison with Standard Geometric Models); the checkpoint is selected on validation. TorchMD-Net ET keeps vector features that transform with rotation, which differs from the distances and scalar value mixing of our models.
  • Global connection, controlled task (only the distance between two fragments changes): local SchNet / DisMix / DisAttention, test MAE↓ = 0.2345±0.00080.2345 \pm 0.0008 / (8.7±2.8)×10−5(8.7 \pm 2.8) \times 10^{-5} / (7.4±0.8)×10−5(7.4 \pm 0.8) \times 10^{-5}.
  • DES370K: each model uses its published default connectivity. For charged dimers beyond 5 Å (mean ∣E∣=2.72|E| = 2.72 kcal/mol), SchNet (10 Å) / DisMix: 0.520.52 / 0.260.26 kcal/mol.
  • Cost: against a channelwise continuous filter in the same backbone, DisMix has a 2.07×2.07\times faster training update on ModelNet40 with 1,024 points.

W3. 12 layers, hidden dimension 384, 12 heads, FFN dimension 384; the same atom, bond, and hop information and RDKit coordinates; official full split; 50 epochs. TGT-Ag, EGT, and GPTrans keep their own pair states and propagation structures. EGT + RDKit / GPTrans + RDKit: 28.73±0.3128.73 \pm 0.31 / 28.37±0.34%28.37 \pm 0.34\%. Relative to Vanilla, DisAttention has about 27% fewer parameters and a 19% shorter step.

W4.

  • Identical node features: if every node value is vv, the output of row-softmax is ∑jPijv=v\sum_j P_{ij} v = v, and this holds across layers that repeat the same node-wise operations. The output of our operator is mivm_i v. The appendix constructs a distinguishable pair of inputs and shows that the difference remains after the residual connection and LayerNorm.
  • Bound on amplification: for a symmetric nonnegative affinity, ∥Z−1/2AZ−1/2∥2=1\lVert Z^{-1/2} A Z^{-1/2} \rVert_2 = 1, and 0<g<20 < g < 2, so ∥W∥2≤2\lVert W \rVert_2 \le 2. The proof and a sensitivity analysis with respect to the log-affinity and the gate are in the appendix.
  • The ModelNet40 control uses the same pairwise distances and backbone and the same training and validation-search budget.

W2. 8 layers, hidden dimension 256, 45 epochs from scratch, the same training configuration. QM9 HOMO MAE↓ in the same order: 33.39±0.5333.39 \pm 0.53 / 31.99±0.0731.99 \pm 0.07 / 31.21±0.3531.21 \pm 0.35 meV. Row-wise softmax fixes each row sum to one; our operator also learns the total transmission.

Q1. Hydrogens are added explicitly; UFF is used when MMFF94 is not applicable. Coordinates are generated once before training and the stored distance matrices are reused. The results in the submission exclude 349 molecules whose generation failed (116 / 97 / 136 in train / validation / test). In the unified comparison these molecules use shortest-path distances, so all models are evaluated on the official full split; the difference in test AP between the two evaluation sets is +0.022+0.022 / +0.015+0.015 %p (pretrained / scratch).

Q2. The node features of LP-PDBbind are the atom type, a protein/ligand indicator, and residue information. A point cloud in ModelNet40 has no separate attribute such as an atom type, so XYZ forms the initial representation of each point.

Q3. DisMix on QM9 ⟨R2⟩\langle R^2 \rangle, 3D → SPD: MAE↓ = 1.788±0.0601.788 \pm 0.060 → 17.218±0.14717.218 \pm 0.147 bohr² (n=3n=3). The MolPCBA comparison uses 30 epochs from scratch and n=2n=2.

Q4.

  • Each head learns the center and width of its Gaussian; b(d)b(d) and the gate take the distance expanded in radial basis functions. The output layers of bb and qq are initialized to zero, so training starts from Gaussian-based mixing with b=0b = 0 and g=1g = 1.
  • A~=Z−1/2AZ−1/2\tilde{A} = Z^{-1/2} A Z^{-1/2} does not change when AA is multiplied by a positive constant. The range (0,2)(0, 2) of gg limits the amplification of each weight to a factor of two.
  • Ablation in the main text, QM9 gap MAE↓: Gaussian + content / without radial bias / without gate / DisMix = 62.68±1.2062.68 \pm 1.20 / 57.64±0.8057.64 \pm 0.80 / 58.28±0.0858.28 \pm 0.08 / 56.29±0.1856.29 \pm 0.18 meV.

Revisions (highlighted in blue in the revised PDF)

  • Related Works: relation to continuous-filter methods and to positional encodings.
  • Distance-based Mixing: function design, role of the gate after normalization, and the interpretation from W=diag(m) ΠW = \mathrm{diag}(m)\,\Pi.
  • MolPCBA experiment: the comparison under unified conditions.
  • Appendix: the theory section Structure, Expressivity, and Stability of Distance-based Mixing, Comparison with Standard Geometric Models, Global Continuous Filters (including DES370K), Additional Controls for the Mixing Operator, Matched MolPCBA Comparison, Distance Inputs and Conformer Sensitivity, and Experimental Details (coordinate generation, node features, RBF design and initialization).

Dear Reviewer Fj1D,

Thank you again for your review. We have posted our response and a revised PDF (changes highlighted in blue). The main additions for each point are as follows.

  • W1. SchNet, EGNN, PaiNN, DimeNet++, and TorchMD-Net ET on QM9 with the same split and budget. DisMix has lower gap error than SchNet and EGNN and higher error than PaiNN, DimeNet++, and TorchMD-Net ET. DES370K charged dimers separated by more than 5 Å, test MAE: PaiNN (5 Å cutoff) / DimeNet++ (5 Å cutoff) / DisAttention = 1.29 / 2.08 / 0.19 kcal/mol.
  • W2. Relative distance bias with the same distance information, changing only the normalization and the gate, QM9 gap MAE: row-softmax / row-softmax + gate / DisMix = 50.56 / 49.35 / 47.83 meV.
  • W3. MolPCBA from scratch with the same backbone, inputs, and budget, test AP: DisAttention / DisMix / Vanilla QK + distance bias = 29.52 ± 0.13 / 29.27 ± 0.22 / 29.22 ± 0.18%; TGT-Ag, EGT, and GPTrans, 28.37–28.75%. DisAttention has 9.47M parameters, the fewest among these models.
  • W4. An appendix theory section (exact decomposition of the operator, the case of identical node features, and a bound on amplification) and a controlled ModelNet40 experiment.
  • Q1–Q4. Coordinate generation, node features per dataset, shortest-path distances, and alternative gate parameterizations are described in the response.

The discussion period ends on November 18 (AoE). We would appreciate knowing whether the response addresses your concerns, and we will answer any remaining questions before then.

Best regards, The Authors

Reviewer_RsR5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes geometry-driven value mixing as an alternative to incorporating spatial distances only as attention biases in graph transformers. The authors introduce DisAttention, which constructs node mixing weights directly from learned functions of pairwise distances without query-key attention, and DisMix, which incorporates node-content affinity into the distance-based mixing operator. Both models apply symmetric normalization followed by a learned distance gate, allowing geometry to control the relative contribution and magnitude of propagated node values without maintaining high-dimensional pair representations. The approach is evaluated on molecular prediction datasets, including QM9, MolPCBA, and LP-PDBbind, as well as MNIST superpixels and ModelNet40. The experiments report competitive predictive performance with fewer parameters and reduced training memory and runtime compared with several graph transformer baselines. Additional ablations and visualizations examine the contributions of radial bias, symmetric normalization, and distance gating. The paper presents distance-driven value mixing as a computationally simpler alternative to more complex attention mechanisms for learning from geometric graphs.

Critical strengths
  1. Interesting perspective on geometry-driven graph learning. The paper addresses an interesting architectural question of whether pairwise geometric distances can directly control how node values are mixed, rather than being used only as additive biases in content-based attention. DisAttention takes a simpler approach by removing query–key attention, while DisMix combines distance-based affinity with node feature information to make the mixing more flexible. Although distance-weighted message passing has been explored in previous work, I think using geometry as the main component of the transformer mixing operator offers a useful alternative perspective on geometric graph learning.

  2. Simple and computationally efficient model design. One positive aspect of the paper is its effort to reduce the computational overhead of maintaining and updating high-dimensional pair representations. DisAttention removes query-key projections and comparisons, while both DisAttention and DisMix avoid repeatedly updating large pair embeddings, as required by some existing geometric graph transformers. I also appreciate that the authors evaluate efficiency beyond parameter counts by reporting actual training step time and peak memory usage. This makes the computational comparison more meaningful and helps demonstrate the practical benefits of the simpler architecture.

  3. Evaluation across different datasets and application domains. The authors evaluate the proposed methods on several datasets, including QM9, MolPCBA, LP-PDBbind, MNIST superpixels, and ModelNet40. I think this is a strength because it shows how the same mixing approach performs across molecular prediction tasks and other geometric learning problems, rather than focusing on a single domain. The comparison between DisAttention and DisMix is also useful, as their different results provide some insight into when distance-based mixing alone may be sufficient and when incorporating additional node affinity can be beneficial.

Critical weaknesses
  1. Limited distinction from existing distance-based message passing. The main operation in DisAttention is distance-dependent weighted aggregation, which is closely related to existing continuous-filter and geometric message-passing methods such as SchNet. Although the proposed approach introduces head-specific distance functions, symmetric normalization, and post-normalization gating, it is unclear how much additional modeling capability these components provide. DisMix also retains query-key interactions, making its distinction from geometry-aware attention mechanisms less clear. A more direct comparison with existing distance-based methods would help establish the methodological contribution.

  2. Performance comparisons are affected by differences in experimental settings. The comparisons with existing graph transformers do not consistently account for differences in pretraining, model capacity, geometric inputs, and training procedures. In particular, Appendix C indicates that DisMix uses pretrained backbone parameters from PCQM4Mv2, while Table 1 includes results from previously published models under different settings. Moreover, 349 MolPCBA molecules are excluded because conformer generation failed, and it is unclear whether the baselines use the same molecular subset. These differences make it difficult to determine how much of the reported performance–parameter advantage comes from the proposed mixing operator itself.

  3. Computational efficiency is not sufficiently evaluated across graph sizes. The reported runtime and memory improvements are encouraging, but the main comparison uses only a batch of four 26-atom molecules. Both DisAttention and DisMix still require dense value mixing with O(N2d)O(N^2d) computation and O(N2H)O(N^2H) storage, while DisMix retains query–key computation. Therefore, its efficiency gains mainly come from avoiding high-dimensional pair-state updates. Although Appendix A discusses these costs, experiments varying graph size, batch size, number of heads, and hidden dimension would provide stronger evidence for the claimed scalability.

  4. Limitations of distance-only geometric expressivity remain unclear. DisAttention constructs mixing weights entirely from pairwise Euclidean distances, which cannot distinguish mirror-image configurations with identical distances and node attributes. This can be important for molecular tasks involving chirality or other orientation-sensitive properties. The operator also does not explicitly encode angles, torsions, or higher-order geometric interactions. While multiple layers may capture some indirect relationships, the paper does not establish which geometric properties the model can represent. Additional experiments on chirality-sensitive or higher-order geometric tasks would help clarify these limitations.

  5. Limited evidence for task-specific geometric reasoning. The learned distance-response visualizations and functional-group interventions provide useful insights, but the conclusions about meaningful molecular interactions rely on a small set of examples. In particular, the MolPCBA analysis focuses on a tertiary-amine-assay combination selected through exploratory analysis, which may introduce selection bias. Although modifying the learned distance responses changes the predictions, this could also result from altering the overall message scale or disrupting information propagation rather than identifying chemically meaningful interactions.

Questions for authors
  1. Novelty of distance-based mixing (Section 2, pp. 2-3; Sections 3.1-3.2, pp. 3-4). The paper proposes constructing node-mixing weights directly from pairwise distances rather than using geometry only as an attention bias. However, distance-dependent weighted aggregation is already used in geometric message-passing methods such as SchNet. Could the authors clarify what makes DisAttention and DisMix methodologically different from these existing approaches? In particular, is the main advantage coming from the normalization and gating strategy, the transformer-based implementation, or the removal of explicit pair-state updates? I think a controlled comparison with established continuous-filter architectures would help make the contribution clearer.

  2. Fairness of comparisons and pretraining (Section 4.1, pp. 5-6; Tables 1-3; Appendix C). The MolPCBA results suggest that DisMix provides a good balance between predictive performance and model size. However, Appendix C mentions that the 13M-parameter model uses pretrained backbone parameters from PCQM4Mv2, while the comparison methods have different architectures, pretraining settings, and training procedures. Could the authors clarify which baselines use comparable pretraining and whether all methods are evaluated on the same molecular cohort? I would also like to understand how much DisMix benefits from pretrained initialization compared with training from scratch. In addition, does excluding molecules for which conformer generation failed affect the fairness of the reported MolPCBA comparisons?

  3. Contribution of geometry-driven mixing (Sections 3.1-3.2, pp. 3-4; Section 4.4; Table 5). Table 5 provides useful ablations showing the effects of the radial bias, distance gate, and symmetric normalization. However, the PlainGT baseline does not use geometric information or bond features, so it is difficult to tell whether the large performance difference comes from the proposed mixing operator or simply from having access to additional structural information. Could the authors compare DisAttention and DisMix with a conventional geometry-aware attention mechanism using the same radial functions, node features, backbone, and training configuration? In particular, would applying the distance gate after normalization still offer an advantage over additive distance biases or gated softmax attention when all other settings are kept the same?

  4. Computational efficiency and scalability (Section 4.1.1, pp. 5-6; Section 5; Appendix A). The reported improvements in training time and memory usage are encouraging, but the main efficiency comparison is based on a small batch of 26-atom molecules. Since both models still rely on dense quadratic mixing, I wonder how well these efficiency gains would hold for larger graphs. Could the authors report how runtime and memory requirements change with the number of nodes, hidden dimension, and batch size? It would also be useful to separate the savings obtained by removing query–key projections in DisAttention from those achieved by avoiding pair-state updates in both models. This would help clarify the graph sizes and computational settings where the proposed approaches provide the greatest advantage.

  5. Geometric expressivity beyond pairwise distances (Sections 3.1-3.2, pp. 3-4; Section 5). DisAttention constructs its mixing weights entirely from pairwise distances, which provides a relatively simple way to incorporate geometry. However, pairwise distances cannot distinguish reflected point configurations and do not directly capture orientation-sensitive geometric information. Could the authors clarify which geometric relationships the proposed operators can represent and where their expressive limitations arise? I am particularly interested in how the models would handle molecular tasks involving chirality, directional interactions, or higher-order angular dependencies. A comparison with directional geometric GNNs would help understand the trade-off between computational efficiency and geometric expressivity.

3 comments

Response to Reviewer RsR5 (1/2)

We thank the reviewer for recognizing the question of whether distance can directly control value mixing, the effort to avoid pair-state updates, and the measurement of actual step time and memory. We also appreciate the specific questions on controlled comparisons and on the limits of expressivity, which the added experiments follow.

W1 / Q1. Difference from continuous-filter methods such as SchNet, and the source of the advantage

Our operator constructs the continuous filter of SchNet as the global value mixing of a graph transformer. The advantage has three sources: the global connection carries relations beyond a cutoff, normalization and the gate control the aggregation scale and the distance-dependent transmission strength, and head-wise scalar sharing with no pair-state update lowers the cost.

  • Global connection: DES370K dimer interaction energies, charged dimers separated by more than 5 Å: PaiNN / DimeNet++ with a 5 Å cutoff, test MAE↓ = 1.291.29 / 2.082.08 kcal/mol; DisAttention, 0.190.19 kcal/mol (n=3n=3).
  • Normalization and gate (QM9 gap MAE↓): DisMix 56.29±0.1856.29 \pm 0.18 meV; without gate 58.28±0.0858.28 \pm 0.08; without radial bias and gate 62.68±1.2062.68 \pm 1.20.
  • Cost: against a SchNet-style channelwise filter in the same backbone (ModelNet40, 1,024 points), training step↓ = 554.8554.8 / 1,150.51{,}150.5 ms (2.07×2.07\times); test accuracy↑ under an equal training budget = 86.63±0.3986.63 \pm 0.39 / 79.63±0.59%79.63 \pm 0.59\%.

Q3. Comparison with geometry-aware attention under the same radial functions, backbone, and training configuration

We replaced only the mixing operator under the same radial functions, node features, and backbone. The gate after normalization gives lower error than an additive distance bias and than gated softmax attention.

  • QM9 gap MAE↓, row-wise softmax / row-wise softmax + gate / DisMix: 50.56±0.2150.56 \pm 0.21 / 49.35±0.1749.35 \pm 0.17 / 47.83±0.5347.83 \pm 0.53 meV (n=3n=3).

W2 / Q2. Pretraining, evaluation cohort, and fairness of comparison

We added a comparison in which all models are trained from scratch with the same backbone, inputs, split, and training budget, and we measured the gain from pretraining and the effect of the excluded molecules separately.

  • MolPCBA from scratch, DisAttention / DisMix / Vanilla QK + distance bias / TGT-Ag: test AP↑ = 29.52±0.1329.52 \pm 0.13 / 29.27±0.2229.27 \pm 0.22 / 29.22±0.1829.22 \pm 0.18 / 28.75±0.16%28.75 \pm 0.16\% (9.479.47 / 13.0213.02 / 12.9612.96 / 13.6313.63M parameters, n=3n=3).
  • Same DisMix, scratch / PCQM4Mv2-pretrained initialization: test AP↑ = 28.30±0.1728.30 \pm 0.17 / 29.82±0.05%29.82 \pm 0.05\% (+1.52±0.22+1.52 \pm 0.22 %p).
  • Change in test AP from excluding the 349 molecules whose conformer generation failed: +0.022+0.022 %p.

W3 / Q4. Efficiency on larger graphs; separating Q/K removal from the omission of pair-state updates

Dense mixing remains quadratic, but the advantage over structures that update high-dimensional pair states grows with graph size.

  • N=128→512N = 128 \to 512: the step-time advantage of DisMix over TGT increases from 1.32×1.32\times to 2.87×2.87\times and the peak-memory ratio from 2.68×2.68\times to 6.29×6.29\times; at N=1,024N = 1{,}024 DisMix runs with 6.06.0 GB and TGT runs out of memory.
  • Effect of removing Q/K (N=512N = 512): DisAttention reduces time by a further 6.2%6.2\% and peak memory by 7.8%7.8\% relative to DisMix.

W4 / Q5. Representable geometric relations, limitations, and comparison with directional GNNs

Angular information is contained in the distance matrix, and the model learns angular relations among several neighbors without an explicit angle operation. Mirror-image structures with the same node features cannot be distinguished, and models that keep directional information obtain lower error on QM9.

  • Target that depends on the angles among 4 neighbors (4 layers), DisAttention / DisMix: test RMSE↓ = (7.03±0.43)(7.03 \pm 0.43) / (5.61±0.50)×10−3(5.61 \pm 0.50) \times 10^{-3} (n=3n=3).
  • QM9 gap MAE↓ (same split, 300 epochs, n=3n=3): DisMix 41.05±0.3641.05 \pm 0.36 meV; PaiNN 40.57±0.0840.57 \pm 0.08; DimeNet++ 35.24±0.2235.24 \pm 0.22; TorchMD-Net ET 34.90±0.3134.90 \pm 0.31.

W5. Selection bias in the functional-group interventions and the effect of message scale

Both points are valid, and we revised the interpretation in the main text. When the weight sum at each receiving node is preserved after the intervention, the effect decreases substantially, so the change in message scale played a large role in the original effect.

  • Decrease in the excess logit for tertiary amines, original intervention → weight sum preserved: 4.59→0.454.59 \to 0.45 (seed 42), 4.25→0.504.25 \to 0.50 (seed 2026).

In summary, the controlled comparisons separate the contributions of the global connection, normalization, and the gate from the cost advantage, and the manuscript now states the chirality limitation and the scope of the functional-group analysis. Conditions, further results, and the list of revisions follow in the next comment, and we are happy to answer any further questions.

Response to Reviewer RsR5 (2/2): conditions, further results, and revisions

W1 / Q1.

  • Global connection, controlled task (only the distance between two fragments changes, 100 epochs, n=3n=3): local SchNet / DisMix / DisAttention, test MAE↓ = 0.2345±0.00080.2345 \pm 0.0008 / (8.7±2.8)×10−5(8.7 \pm 2.8) \times 10^{-5} / (7.4±0.8)×10−5(7.4 \pm 0.8) \times 10^{-5}. ModelNet40 (256 points, constant node features), local / global DisMix: 49.97±0.9949.97 \pm 0.99 / 74.80±1.36%74.80 \pm 1.36\%.
  • DES370K: each model uses its published default connectivity. For charged dimers beyond 5 Å (mean ∣E∣=2.72|E| = 2.72 kcal/mol), SchNet (10 Å) / DisMix: 0.520.52 / 0.260.26 kcal/mol.
  • Training stability (QM9 gap, fixed learning rates 2×10−32 \times 10^{-3} and 3×10−33 \times 10^{-3}, n=3n=3): without normalization 3/3 runs diverge; DisMix completes 3/3.
  • Aggregation scale: accuracy change when checkpoints trained with 256 points are applied to 1,024 points, DisAttention / DisMix / channelwise Transformer = −1.99±2.31-1.99 \pm 2.31 / −4.57±5.60-4.57 \pm 5.60 / −28.73±6.51-28.73 \pm 6.51 %p.
  • Remaining components: without normalization 57.48±1.0757.48 \pm 1.07, without radial bias 57.64±0.8057.64 \pm 0.80 meV.
  • Channelwise-filter comparison: 4 layers, hidden dimension 128, 4 heads (4 filter outputs per pair against 128), an equal training budget equivalent to 3,600 s.
  • MolPCBA with the same inputs and hardware (RTX 5090, BF16): step time↓, DisAttention / Vanilla = 22.9722.97 / 28.3028.30 ms; DisMix / TGT-Ag = 28.7628.76 / 44.8544.85 ms, peak memory↓ = 2.22.2 / 5.25.2 GB.

Q3. 8 layers, width 256, the same atom, bond, and 3D inputs and split, 45 epochs from scratch. PlainGT in Table 5 is an auxiliary control without distances or bond features; the main text now states this input difference. The comparison with a Vanilla model using raw QK + distance bias is the MolPCBA result in W2 / Q2.

W2 / Q2.

  • Conditions: 12 layers, hidden dimension 384, 12 heads, FFN dimension 384; the same atom, bond, and hop information and RDKit coordinates; official full split; 50 epochs. EGT + RDKit / GPTrans + RDKit: 28.73±0.3128.73 \pm 0.31 / 28.37±0.34%28.37 \pm 0.34\% (13.1713.17 / 14.9514.95M).
  • The pretraining comparison uses 30 downstream epochs and the same 2 seeds.
  • The 349 excluded molecules are 116 / 97 / 136 in train / validation / test (43,657 of the 43,793 official test molecules are evaluated). In the matched comparison, failed molecules use shortest-path distances, so all models are evaluated on the official full split. The AP change for the scratch model is +0.015+0.015 %p.

W3 / Q4.

  • On LP-PDBbind complexes with 4,000–4,700 nodes, DisMix has a 5.85×5.85\times faster update than a channelwise filter in the same backbone.
  • Measurements that vary batch size, hidden dimension, and the number of heads, and results on ModelNet40 with N=256N = 256 / 512512 / 1,0241{,}024, are in the appendix Computational Efficiency.
  • A vanilla attention is faster and smaller in some of these settings, but its accuracy is much lower on some datasets and not consistent across domains (PlainGT / DisMix: QM9 gap MAE 793.43793.43 / 187.14187.14 meV, LP-PDBbind RMSE 1.6611.661 / 1.5471.547; MNIST superpixel accuracy, PlainGT / DisAttention: 97.5797.57 / 98.90%98.90\%).

W4 / Q5.

  • 8 neighbors (4 layers), DisAttention / DisMix: (7.14±0.77)(7.14 \pm 0.77) / (6.48±1.23)×10−3(6.48 \pm 1.23) \times 10^{-3}. The inputs are the full distance matrix and node features that mark the center and the neighbors.
  • In the same QM9 comparison, SchNet 51.85±0.5651.85 \pm 0.56 and EGNN 44.64±1.6244.64 \pm 1.62 meV. TorchMD-Net ET keeps vector features that transform with rotation.
  • A distance representation invariant to rotation and translation suits the scalar prediction tasks of this work; chirality-sensitive tasks additionally need features that distinguish mirror images or operations that process directional information.

W5. The validation data are split by Murcko scaffold into exploration and confirmation sets; we report all three fixed functional groups (tertiary amine, urea, phenol) with atom-count-matched controls. For urea the effect also decreases substantially when the weight sum is preserved; for phenol it is small and its direction depends on the seed.

Revisions (highlighted in blue in the revised PDF)

  • Related Works, Distance-based Mixing: relation to continuous-filter methods; roles of head-wise scalar sharing, normalization, and the gate.
  • MolPCBA experiment: the matched scratch comparison, the gain from pretraining, and the evaluation scope of each result.
  • Runtime and Memory: change of the advantage with size and the conditions under which it holds.
  • Ablation of the Mixing Operator: input difference of PlainGT and the operator-replacement comparison.
  • Learned Distance Responses and Mixing Patterns: interpretation of the functional-group interventions.
  • Discussion and Limitations: angular dependence, scope of invariance, and the chirality limitation.
  • Appendix: the conditions and full results of the experiments above (Global Continuous Filters includes DES370K) and the theory section.

Dear Reviewer RsR5,

Thank you again for your review. We have posted our response and a revised PDF (changes highlighted in blue). The main additions for each point are as follows.

  • W1 / Q1, Q3. Operator replacement in the same backbone, QM9 gap MAE: row-softmax / row-softmax + gate / DisMix = 50.56 / 49.35 / 47.83 meV. DES370K charged dimers separated by more than 5 Å, test MAE: PaiNN (5 Å cutoff) / DimeNet++ (5 Å cutoff) / DisAttention = 1.29 / 2.08 / 0.19 kcal/mol.
  • W2 / Q2. MolPCBA from scratch with the same backbone, inputs, and budget, test AP: DisAttention / DisMix / Vanilla QK + distance bias = 29.52 ± 0.13 / 29.27 ± 0.22 / 29.22 ± 0.18%. The gain from pretraining and the effect of the 349 excluded molecules are reported separately.
  • W3 / Q4. From N = 128 to 512, the step-time ratio of TGT to DisMix increases from 1.32× to 2.87× and the peak-memory ratio from 2.68× to 6.29×. The effect of removing Q/K is reported separately.
  • W4 / Q5. An angle-dependent task, a statement of the chirality limitation, and a QM9 comparison in which PaiNN, DimeNet++, and TorchMD-Net ET have lower error than DisMix.
  • W5. When the weight sum is preserved, the intervention effect decreases substantially; we revised the interpretation in the main text.

The discussion period ends on November 18 (AoE). We would appreciate knowing whether the response addresses your concerns, and we will answer any remaining questions before then.

Best regards, The Authors

Reviewer_C8NcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DisAttention, which replaces query–key attention with learned distance-based value mixing, and DisMix, which additionally incorporates node-feature affinity. Both use symmetric normalization and distance gating to achieve competitive prediction while reducing computational overhead relative to graph transformers with high-dimensional pair representations.

Critical strengths
  1. The paper presents a simple geometry-driven mixing mechanism that reduces computational overhead while retaining competitive predictive performance.

  2. Evaluation across molecular and nonmolecular datasets, together with operator ablations, provides evidence of the approach’s versatility and the contributions of distance gating and normalization.

Critical weaknesses
  1. The framework assumes access to meaningful node coordinates, limiting its demonstrated applicability to general graphs. Its behavior on graphs without natural coordinates remains unclear. For molecular applications, evaluating sensitivity to generated conformers and including geometry-generation costs would better establish practical applicability and efficiency.

  2. MolPCBA comparisons involve models with different parameter counts, pretraining, and input-processing pipelines. Comparisons within the same backbone, using matched training budgets and inputs, would better isolate gains attributable to the proposed operator.

  3. The LP-PDBbind evaluation protocol is insufficiently specified. Although Table 3 reports binding-affinity results, the paper does not describe the dataset release, train/validation/test split, sample counts, or protein–ligand preprocessing. These omissions limit reproducibility and make it difficult to assess potential data leakage and the fairness of the baseline comparisons.

  4. The paper acknowledges that existing methods already derive interaction weights from distances. Controlled comparisons with continuous-filter and other geometric message-passing approaches would help establish the added value of its normalization and gating design.

  5. DisMix substantially improves over DisAttention on MolPCBA, but performs worse on four of six QM9 targets and on ModelNet40. More analysis is needed to establish when content affinity helps and why it can degrade performance.

Questions for authors
  1. How would the framework handle graphs without natural coordinates? How sensitive are molecular results to conformer choice, and do efficiency measurements include geometry generation?

  2. Can the authors compare mixing operators within the same backbone using matched inputs, pretraining, parameter counts, and training budgets?

  3. Which LP-PDBbind release, split, sample counts, and protein–ligand preprocessing were used? Were these settings identical across baselines?

  4. What benefits do the proposed normalization and gating provide over continuous-filter and other distance-based message-passing methods under controlled comparisons?

  5. Why does content affinity improve MolPCBA performance but degrade several QM9 targets and ModelNet40? Under what conditions should DisMix be preferred over DisAttention?

4 comments

Response to Reviewer C8Nc (1/2)

We thank the reviewer for the positive assessment of the simple geometry-driven mixing mechanism and of the evaluation across molecular and non-molecular datasets. We also appreciate the concrete suggestions on comparison conditions and protocols, which we followed in the added experiments.

W2 / Q2. Comparison within the same backbone, inputs, and training budget

We trained all models from scratch with the same node backbone, the same inputs, and the same 50-epoch budget. Under these conditions DisAttention obtains the highest mean AP with the fewest parameters.

  • DisAttention / DisMix / Vanilla QK + distance bias / TGT-Ag: test AP↑ = 29.52±0.1329.52 \pm 0.13 / 29.27±0.2229.27 \pm 0.22 / 29.22±0.1829.22 \pm 0.18 / 28.75±0.16%28.75 \pm 0.16\% (9.479.47 / 13.0213.02 / 12.9612.96 / 13.6313.63M parameters, n=3n=3).

W4 / Q4. Benefit over continuous-filter and distance-based message passing

Our operator constructs the continuous filter of SchNet as the global value mixing of a graph transformer. Head-wise scalar weights with normalization and gating implement this global filter at a lower cost, and the global connection carries interactions beyond a cutoff.

  • Against a SchNet-style channelwise filter in the same backbone (ModelNet40, 1,024 points): training step↓ = 554.8554.8 / 1,150.51{,}150.5 ms (2.07×2.07\times); test accuracy↑ under an equal training budget = 86.63±0.3986.63 \pm 0.39 / 79.63±0.59%79.63 \pm 0.59\%.
  • DES370K dimer interaction energies, charged dimers separated by more than 5 Å: PaiNN / DimeNet++ with a 5 Å cutoff, test MAE↓ = 1.291.29 / 2.082.08 kcal/mol; DisAttention, 0.190.19 kcal/mol (n=3n=3).

W1 / Q1. Graphs without coordinates, conformer sensitivity, and geometry-generation cost

Graphs without coordinates are outside the scope of this work, but the same operator can be applied to shortest-path distances, and the results are stable with respect to the choice of generated conformer. The runtime in the submission does not include coordinate generation; we now also report the cost with it included.

  • Change in MolPCBA test AP with two independently regenerated RDKit conformers, DisMix: +0.050±0.047+0.050 \pm 0.047 / +0.091±0.027+0.091 \pm 0.027 %p (n=3n=3).

W5 / Q5. When content affinity helps

Content affinity modulates the transmission proportions by node states even for the same geometric relation, and its gain is clear at limited depth. The degradation observed in the submission does not appear consistently in repeated comparisons under matched conditions.

  • Angle-dependent target, 1 layer, DisAttention / DisMix: test MAE↓ = (5.51±0.78)(5.51 \pm 0.78) / (3.54±0.23)×10−3(3.54 \pm 0.23) \times 10^{-3} (n=5n=5); with 4 layers, (1.51±0.06)(1.51 \pm 0.06) / (1.48±0.42)×10−3(1.48 \pm 0.42) \times 10^{-3}.

W3 / Q3. LP-PDBbind protocol

We use PDBbind v2020 structures with the CL1 noncovalent complexes and the released new_split of LP-PDBBind. All baselines use the same complexes, split, preprocessing, and targets.

  • Train / validation / test = 7,3937{,}393 / 1,8911{,}891 / 4,2504{,}250; PDB IDs shared between splits = 00 / 00 / 00.

In summary, under matched conditions the proposed operator reaches equal or higher performance with fewer parameters and lower cost, and the points on coordinates and protocols are addressed with added experiments and explicit descriptions. Conditions, further results, and the list of revisions follow in the next comment, and we are happy to answer any further questions.

Response to Reviewer C8Nc (2/2): conditions, further results, and revisions

W2 / Q2. 12 layers, hidden dimension 384, 12 heads, FFN dimension 384; the same atom, bond, and hop information and RDKit coordinates; official split. TGT-Ag, EGT, and GPTrans keep their own pair states and propagation structures.

  • EGT + RDKit / GPTrans + RDKit: test AP↑ = 28.73±0.3128.73 \pm 0.31 / 28.37±0.34%28.37 \pm 0.34\% (13.1713.17 / 14.9514.95M).
  • Same inputs and hardware (RTX 5090, BF16): step time↓, DisAttention / Vanilla = 22.9722.97 / 28.3028.30 ms; DisMix / TGT-Ag = 28.7628.76 / 44.8544.85 ms, peak memory↓ = 2.22.2 / 5.25.2 GB.

W4 / Q4.

  • Global connection, controlled task (only the distance between two fragments changes): local SchNet / DisMix / DisAttention, test MAE↓ = 0.2345±0.00080.2345 \pm 0.0008 / (8.7±2.8)×10−5(8.7 \pm 2.8) \times 10^{-5} / (7.4±0.8)×10−5(7.4 \pm 0.8) \times 10^{-5}. ModelNet40 (256 points, constant node features), local / global DisMix: 49.97±0.9949.97 \pm 0.99 / 74.80±1.36%74.80 \pm 1.36\%.
  • Global connection, real data (added): DES370K dimer interaction energies, each model with its published default connectivity, n=3n=3. For charged dimers separated by more than 5 Å (mean ∣E∣=2.72|E| = 2.72 kcal/mol), PaiNN (5 Å) / DimeNet++ (5 Å) / SchNet (10 Å) / DisAttention / DisMix: test MAE↓ = 1.291.29 / 2.082.08 / 0.520.52 / 0.190.19 / 0.260.26 kcal/mol.
  • Aggregation scale: accuracy change when checkpoints trained with 256 points are applied to 1,024 points, DisAttention / DisMix / channelwise Transformer = −1.99±2.31-1.99 \pm 2.31 / −4.57±5.60-4.57 \pm 5.60 / −28.73±6.51-28.73 \pm 6.51 %p.
  • Components (QM9 gap MAE↓, meV): DisMix 56.29±0.1856.29 \pm 0.18; without normalization 57.48±1.0757.48 \pm 1.07; without radial bias 57.64±0.8057.64 \pm 0.80; without gate 58.28±0.0858.28 \pm 0.08; without radial bias and gate 62.68±1.2062.68 \pm 1.20.
  • Other distance-based and geometric message-passing models (QM9 gap, same split, 300 epochs, n=3n=3, meV): SchNet 51.85±0.5651.85 \pm 0.56, EGNN 44.64±1.6244.64 \pm 1.62, DisMix 41.05±0.3641.05 \pm 0.36, PaiNN 40.57±0.0840.57 \pm 0.08, DimeNet++ 35.24±0.2235.24 \pm 0.22, TorchMD-Net ET 34.90±0.3134.90 \pm 0.31. DisMix is below SchNet and EGNN and above the models that keep directional information.

W1 / Q1.

  • 3D → shortest-path distances: DisMix on QM9 gap, MAE↓ = 56.19±0.2256.19 \pm 0.22 → 91.22±0.6991.22 \pm 0.69 meV; DisAttention on MolPCBA, test AP↑ = 29.35±0.2029.35 \pm 0.20 / 28.81±0.12%28.81 \pm 0.12\% (scratch, 30 epochs, n=2n=2).
  • Conformers, DisAttention: −0.057±0.089-0.057 \pm 0.089 / +0.055±0.133+0.055 \pm 0.133 %p.
  • Coordinate generation for all 437,929 MolPCBA molecules: 71.7571.75 min with 8 CPU workers (8.2258.225 CPU hours), paid once and reused. 50-epoch training time including it: DisAttention 124.2124.2, DisMix 137.4137.4, TGT-Ag + RDKit 174.1174.1 min.

W5 / Q5.

  • Angle task, 2 layers: (2.11±0.27)(2.11 \pm 0.27) / (2.17±0.55)×10−3(2.17 \pm 0.55) \times 10^{-3}.
  • QM9 gap (same backbone, 45 epochs, n=3n=3), DisAttention / DisMix: 48.82±0.0648.82 \pm 0.06 / 47.83±0.5347.83 \pm 0.53 meV.
  • MolPCBA, initial content coefficient λ0=0\lambda_0 = 0 / 0.950.95: 30 epochs 29.13±0.3529.13 \pm 0.35 / 28.56±0.30%28.56 \pm 0.30\%; 100 epochs 29.45±0.3029.45 \pm 0.30 / 29.35±0.17%29.35 \pm 0.17\%.
  • ModelNet40 (256 points, 40 epochs, n=3n=3), DisAttention / DisMix: 86.98±0.6386.98 \pm 0.63 / 87.43±0.16%87.43 \pm 0.16\%. The comparison in the submission used a single seed.
  • Choice: DisAttention when reducing Q/K computation matters; DisMix when relations need to be modulated by node states at limited depth. In practice the choice can be made from validation results under a common training condition together with the cost.

W3 / Q3. Preprocessing and node features are specified in Experimental Details. Identical protein-sequence groups between splits = 1313 / 4646 / 1414 and identical ligand-SMILES groups = 00 / 00 / 133133 (train–validation / train–test / validation–test); 599 test complexes have a protein sequence that also appears in train.

Revisions (highlighted in blue in the revised PDF)

  • MolPCBA experiment: the matched scratch comparison and the evaluation scope.
  • Related Works, Distance-based Mixing: relation to continuous-filter methods; roles of head-wise scalar sharing, normalization, and the gate; summary of the local/global comparison.
  • Runtime and Memory: the measurements cover the cost after coordinate preparation.
  • DisMix, Discussion and Limitations: role of content affinity and the conditions under which it helps.
  • Appendix: Distance Inputs and Conformer Sensitivity, Matched MolPCBA Comparison, Global Continuous Filters (including DES370K), Role of Content Affinity, Comparison with Standard Geometric Models, Computational Efficiency, Experimental Details (LP-PDBbind protocol, coordinate sources).
Reviewer_C8NcOpenReview ↗

Response to Author

Thank you for the detailed response. However, my concern has been only partially resolved. Since the model depends on node coordinates, its general applicability to other domains such as social networks and brain networks remains limited. As most molecular datasets come with natural node coordinates (atoms), the paper appears more suitable for molecular representation learning than for general graph learning. Therefore, I will keep my score.

Response to Reviewer C8Nc (follow-up)

Thank you for your prompt reply. We agree that the method requires a meaningful pairwise distance, and that graphs without such distances, including social and brain networks, are not evaluated in this work. We will state this scope in the abstract, introduction, and limitations of the final version.

For reference, the evaluated data are small molecules (QM9, MolPCBA), protein–ligand complexes (LP-PDBbind), intermolecular interactions (DES370K), 3D point clouds (ModelNet40), and 2D image superpixels (MNIST superpixel). With shortest-path distances in place of Euclidean distances, trained from scratch, DisMix obtains a QM9 gap MAE of 91.22 ± 0.69 meV against 56.19 ± 0.22 meV with 3D distances, and DisAttention a MolPCBA test AP of 28.81 ± 0.12% against 29.35 ± 0.20% (Appendix D, Table 12). The operator can be applied to graph distances, and the effect of removing continuous geometry depends on the task; these results are on molecular graphs.

Best regards, The Authors

Comments on the paper

1 comment

General Response to All Reviewers and the AC

We thank the three reviewers for their detailed reviews. The reviewers recognized the question of using distance as a direct value-mixing operator instead of an attention bias (RsR5, Fj1D), the simple design that avoids high-dimensional pair-state updates together with the measurement of actual step time and memory (C8Nc, RsR5, Fj1D), and the evaluation across molecular and non-molecular datasets (C8Nc, RsR5). The shared concern was the lack of controlled comparisons. We summarize the added experiments and the changes to the manuscript below; detailed answers are given in the response to each reviewer.

Position of the work. Our work shares the continuous-filter view of SchNet and constructs it as the global value mixing of a graph transformer. Symmetric normalization handles the aggregation scale, and the distance gate after normalization handles the distance-dependent transmission strength. Head-wise scalar sharing, the removal of Q/K in DisAttention, and the omission of high-dimensional pair-state updates connect to computational efficiency.

Main added experiments.

  1. MolPCBA from scratch with the same backbone, inputs, and training budget (3 seeds): DisAttention / DisMix / Vanilla QK + distance bias, test AP↑ = 29.52 ± 0.13 / 29.27 ± 0.22 / 29.22 ± 0.18%; TGT-Ag, EGT, and GPTrans obtain 28.37–28.75%. DisAttention has the highest mean AP with 9.47M parameters.
  2. Role of the global connection: on a controlled fragment task, local SchNet has a test MAE of 0.2345, while DisMix and DisAttention reach 8.7 × 10⁻⁵ and 7.4 × 10⁻⁵. On DES370K dimer interaction energies, for charged dimers separated by more than 5 Å, PaiNN / DimeNet++ with a 5 Å cutoff: test MAE↓ = 1.29 / 2.08 kcal/mol; DisAttention: 0.19 kcal/mol (3 seeds). On ModelNet40, local / global DisMix: accuracy↑ = 49.97 ± 0.99 / 74.80 ± 1.36%.
  3. Operator replacement with the same inputs on QM9 gap: row-softmax / row-softmax + gate / DisMix = 50.56 ± 0.21 / 49.35 ± 0.17 / 47.83 ± 0.53 meV.
  4. Standard geometric models on QM9 gap (300 epochs): SchNet, EGNN, PaiNN, DimeNet++, and TorchMD-Net ET were evaluated under the same conditions. DisMix has lower error than SchNet and EGNN and higher error than the models that keep directional information. Among the graph transformers compared in the paper (Table 2), DisAttention and DisMix have the lowest and second-lowest training step time and peak memory: 19.18 / 37.20 ms and 0.141 / 0.308 GiB, against 43.78–84.83 ms and 1.005–5.219 GiB for the others.
  5. Efficiency with graph size: from N = 128 to 512, the step-time advantage of DisMix over TGT increases from 1.32× to 2.87× and the peak-memory ratio from 2.68× to 6.29×.
  6. Coordinates and evaluation protocol: we report the replacement of Euclidean by shortest-path distances, conformer sensitivity, the cost of coordinate generation, the LP-PDBbind protocol with an overlap check between splits, and the MolPCBA evaluation sets.

Changes to the manuscript. Changes are highlighted in blue in the revised PDF. The main text keeps its length and now includes: the relation to continuous-filter methods and to positional encodings (Section 2); head-wise scalar sharing and the decomposition W = diag(m) Π (Section 3); the matched-backbone comparison, the gain from pretraining, and the evaluation scope (Section 4.1); efficiency with size and the conditions under which the advantage holds (Section 4.1.1); the operator-replacement comparison and the input difference of PlainGT (Section 4.4); the interpretation of the functional-group interventions (Section 4.5); and the role of content affinity, angular dependence, and the chirality limitation (Section 5). The abstract is revised accordingly. The appendix adds a theory section, Structure, Expressivity, and Stability of Distance-based Mixing, and the conditions and full results of the experiments above.

We thank the reviewers and the AC again and welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.