acceptodds
Under review as a conference paper at ICLR 2027

MoDANet: A Motif-Aware Distance-Masked Attention Network for Molecular Property Prediction

Abstract

Reliable molecular property prediction is essential for drug discovery, yet standard graph Transformers use unrestricted global attention without explicitly organizing atom interactions by topological range. We introduce Motif-Aware Distance-Masked Attention Network (MoDANet), a dual-branch multi-scale graph Transformer whose global branch constructs fixed, nested, and permutation-equivariant neighborhoods from shortest-path distances and learns a layer-wise convex mixture over local, intermediate, and global attention scopes. Its motif branch directly encodes functional-group fragments obtained through BRICS decomposition. MoDANet provides an inspectable, distance-structured attention mechanism, is evaluated under scaffold splits on seven MoleculeNet benchmarks, and achieves state-of-the-art transfer performance after pretraining on a 250K-molecule ZINC subset.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph transformers, molecular property prediction, attention mechanism, self-supervised learning, graph neural networks

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_csdgOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MoDANet, a dual-branch multi-scale graph Transformer for molecular property prediction. It targets two weaknesses in current atom-level models. First, global self-attention treats all atoms alike: a distant atom counts as much as a bonded one. Second, explicit substructure hierarchy is missing. MoDANet pairs a global branch with a motif branch. The global branch learns a convex mixture over nested, coverage-based shortest-path scopes at each layer. The motif branch works on BRICS motifs and needs no closed fragment vocabulary. The two views are fused by global-context-guided motif aggregation. The main contributions are a distance-masked attention mechanism with learnable per-layer scope mixtures, and a BRICS-based motif branch with no fragment vocabulary.

Critical strengths

The method combines a learnable distance-masked attention mechanism with a BRICS fragment branch that needs no vocabulary. At each layer, attention is a convex mixture over topological scopes. These mixtures can be inspected directly. The approach needs no graph augmentation and no 3D conformers.

Critical weaknesses

The state-of-the-art claim is not adequately supported. Gains over the strongest baselines are small, and they do not hold consistently across datasets. Several baselines were also pretrained on larger or richer corpora. Without experiments that control for pretraining-set size, the sample-efficiency claim remains unverified. The method is moreover close to an incremental combination of existing components.

Questions for authors

The SOTA claim is weaker than stated. MoDANet is best on only 4 of 7 benchmarks, with margins of 0.7 to 3.2 points, and falls behind on Tox21 and MUV. Please report whether these differences are statistically significant. For MUV the gap to SimSGT is 1.9 points; is that difference within run-to-run variance? The controlled comparison is still missing. The authors acknowledged that several baselines use a larger or a richer pretraining corpora, and that controlled scaling experiments are needed. The central claim, however, is that SOTA can be reached with only a 250K-molecule ZINC subset. Please provide at least one controlled experiment in which a strong baseline is pretrained on the identical 250K ZINC subset and fine-tuned under the same scaffold splits and budget. Otherwise the reader cannot separate architecture gains from differences in data and objective.

2 comments

Response to Q2:

We thank the reviewer for this suggestion. We have retrained SimSGT on ZINC 250K under the same setting as MoDANet.

DatasetMoDANetSimSGT (controlled)
BBBP74.7±0.471.4±3.8
ClinTox94.4±0.978.0±1.5
MUV77.0±0.976.9±0.4
HIV79.3±0.877.5±0.9
BACE87.0±1.781.2±0.3
Tox2177.6±0.176.4±0.8
SIDER64.8±0.262.3±1.5

Under this controlled setting, MoDANet performs better on all seven benchmarks, with MUV essentially tied. On Tox21, MoDANet now leads (77.6 vs. 76.4), whereas it trailed the published result obtained with a larger pretraining corpus.

Response to Q1:

We thank the reviewer for these comments. The pretraining-data concern is addressed by the controlled experiment in Q2: with SimSGT retrained under the same setting (ZINC 250K), MoDANet still performs better on all seven benchmarks, with MUV essentially tied.

Regarding incremental combination, we conducted an additive ablation in which every variant is pretrained independently from scratch under the same setting (ZINC 250K):

Datasetbase+VDAA+motif+motif+GCMA+VDAA+motiffull
BBBP63.8±3.168.2±0.768.0±1.771.5±2.371.6±0.774.7±0.4
ClinTox77.6±2.086.9±1.681.0±1.985.1±1.891.5±1.494.4±0.9
BACE68.3±2.779.8±0.981.4±1.181.6±0.982.8±0.887.0±1.7
Tox2172.0±0.273.6±0.474.0±0.374.2±0.774.5±1.077.6±0.1
SIDER52.1±3.353.6±3.354.1±3.655.2±2.556.7±1.764.8±0.2

The full model performs best on all five benchmarks. Adding GCMA on top of +VDAA+motif brings consistent gains (82.8→87.0 on BACE, 56.7→64.8 on SIDER, 74.5→77.6 on Tox21), and VDAA improves both the base model and the motif-only variant. The gains therefore come from how the components work together, not from simply stacking them.

Thanks for your responses. My concerns are resolved. But I still think that the main contribution is combination of existing components.

Reviewer_BLThOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

MoDANet is a dual-branch graph Transformer for molecular property prediction. The global branch constructs three nested attention scopes from shortest-path distances, computes attention separately within each scope, and then combines them using layer-wise learned weights. The motif branch decomposes molecules with BRICS, directly encodes the atom-level graph of each fragment, and then aggregates the fragment representations using the global molecular representation as the query. The model is pretrained on approximately 250,000 ZINC molecules by predicting a computed penalized logP target, and is subsequently fine-tuned on downstream tasks.

Critical strengths
  • Molecular properties often depend on local functional groups, broader topological relationships, and the overall molecular context. Modeling these multiple structural scales is therefore well motivated.
  • The method integrates local topology, long-range relationships, and functional fragments within a single model.
  • The BRICS-based motif branch directly encodes functional-group fragments from atom-level graphs and fuses them with the global representation, without relying on a closed fragment vocabulary.
  • On the classification tasks, the model achieves strong results among the reported baselines on BBBP, ClinTox, BACE, and HIV.
Critical weaknesses
  • VDAA is effectively a weighted combination of attention distributions computed under several nested distance masks. The mixing weights are a single vector for each layer, shared across all molecules and attention heads. Thus, the method is layer-adaptive rather than input-adaptive in its selection of attention ranges. The paper should describe its claims about dynamically selecting topological scopes more precisely.
  • VDAA still computes attention over all node pairs and repeats the softmax operation under multiple masks. The shortest-path matrix also has to be precomputed. Therefore, the method does not provide sparse attention or lower computational complexity, and may instead increase computational and memory costs.
  • The motif branch is trained mainly through the single scalar penalized logP objective. It is therefore unclear whether the branch actually learns stable functional-fragment representations.
  • Some performance drops in Table 2 are very large, but the paper does not state whether each ablated variant was pretrained from scratch, retuned, or subjected to a new search over the attention scopes.
Questions for authors
  1. Were all baselines retrained, and were they evaluated with consistent data splits, input features, and hyperparameter budgets?
  2. Could the authors provide comparisons with fixed uniform mixing, shared mixing weights across layers, fixed hop-based ranges, and parameter-matched models?
5 comments

Q1:VDAA is effectively a weighted combination of attention distributions computed under several nested distance masks. The mixing weights are a single vector for each layer, shared across all molecules and attention heads. Thus, the method is layer-adaptive rather than input-adaptive in its selection of attention ranges. The paper should describe its claims about dynamically selecting topological scopes more precisely. A1:Thank you for the precise observation. You are right that the scope-mixing weights are layer-wise vectors shared across molecules and heads, so the mechanism is layer-adaptive, not input-adaptive. We will revise the wording to "layer-wise adaptive scope mixing" to avoid overstating the adaptivity.

Q2:VDAA still computes attention over all node pairs and repeats the softmax operation under multiple masks. The shortest-path matrix also has to be precomputed. Therefore, the method does not provide sparse attention or lower computational complexity, and may instead increase computational and memory costs.

A2:Thank you for pointing this out. You quite right that VDAA is not a sparse-attention mechanism and does not reduce asymptotic complexity; that was never our design goal. VDAA introduces inductive bias into the model rather than pursuing computational efficiency. From a complexity standpoint, VDAA does not change order of attention. Standard all-pairs attention costs O(N²) in both time and memory per head; VDAA simply repeats this under K nested distance masks, giving O(K·N²) time and O(N²) memory, where K=3 a fixed constant. The shortest-path matrix is precomputed once per molecular graph, and molecular graphs contain only tens of nodes, so this preprocessing is negligible relative to training. We emphasize that VDAA is not intended as sparse attention or as a complexity reduction; the is a constant factor K on top of standard attention, not an increase in asymptotic order. We will include this complexity analysis explicitly in the revised paper.

Q3:The motif branch is trained mainly through the single scalar penalized logP objective. It is therefore unclear whether the branch actually learns stable functional-fragment representations.

A3:Thank you for raising this concern. Although the reported model is pretrained solely with the penalized-logP regression objective, the motif branch does not learn fragments without structural constraints. Its inputs are deterministically obtained through BRICS decomposition, providing explicit chemical anchors for the resulting fragment representations. Pretraining initializes this branch, while subsequent end-to-end fine-tuning further optimizes the fragment representations under downstream task supervision. The ablations support the practical contribution of this branch. Removing the motif branch decreases performance by 8.8 points on BBBP, 41.8 on ClinTox, 10.0 on BACE, 4.7 on Tox21, and 14.2 on SIDER, with consistent performance drops across all five datasets. These results indicate that the motif branch learns and exploits fragment information relevant to downstream molecular properties, rather than serving as an unative add-on. We will therefore describe the learned representations as task-relevant functional-fragment representations. At the same time, we agree that these ablations primarily demonstrate downstream utility, and we will not present them alone as direct evidence for the stability of the fragment representations.

Q4:Some performance drops in Table 2 are very large, but the paper does not state whether each ablated variant was pretrained from scratch, retuned, or subjected to a new search over the attention scopes.

A4:Thank you for pointing out missing detail. Except for the w/o pretrain control, all architectural ablated variants were fine-tuned from the same pretrained checkpoint. They were not separately pretrained and did not involve a new search over attention scopes or per-variant hyperparameter retuning. The hyperparameters were fixed per dataset and shared across all architectural variants. The w/o pretrain row is the explicit exception, where the same architecture is trained without pretrained weights. We will add this experimental-protocol clarification to the appendix in the revised paper.

Q5:Were all baselines retrained, and were they evaluated with consistent data splits, input features, and hyperparameter budgets?

A5:Most baselines were evaluated by fine-tuning the official pretrained checkpoints released by each method, while a small number retrained by us when no official checkpoint was available or verification was needed. All models were compared under an identical evaluation protocol: the molmcl scaffold 80/10/10 split , unified input featurization (OGB atom/b features with RWPE, with 3D models using their own conformers), three independent seeds reported as mean and standard deviation, and literature-aligned metrics (ROC-AUC for classification; RMSE for ESOL/FreeSolv/Lipophilicity; MAE for QM7/QM8/QM9, with QM9 being the homo/lumo/gap tasks inree).

Thank you for the detailed response. The clarifications address several of my initial concerns. In particular, the authors have clarified that VDAA is layer-adaptive rather than input-adaptive, and the motif branch is intended to improve downstream performance rather than ensure representation stability.

My main remaining concern is the ablation protocol. Since the architectural variants were fine-tuned from the full model's pretrained checkpoint instead of being pretrained independently, the substantial performance drops reported in Table 2 do not clearly demonstrate the individual contributions of each component.

Overall, the response addresses several important issues, and I believe the remaining concern can be addressed with further clarification and a more careful discussion of the ablation results. Given the reasonable motivation behind the method and its strong empirical results, I am inclined to raise my score to weak accept.

Reviewer_q8YPOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

I thank the authors for the interesting paper, they present a new dual branch graph transformer for molecular property prediction called MoDANet. This method combines graph attention with masked masked distance matrices, with retro-synthetic motifs in a separate encoder. This is well motivated by including additional chemically meaningful information into the model trunk. The authors show strong results across a range of benchmarks, and compare results cleanly against a range of competitor models.

Overall I really liked the paper, and with a few small improvements will be a good addition to the field.

Critical strengths

The dual-branch transformer is a clever introduction that allows the incorporation of otherwise absent information about which groups are functional, and where bonds break / form. While with enough data one might expect this is to be learnable from the data, at the scale of the Zinc dataset these are at best patterns rather than rules, so this method allows the model to include additional inductive biases very neatly.

The ablations start to show the relative importance of all the various components of the architecture, I like this a lot - taking this further would make this a very strong paper. (See suggestions below)

The paper is well written, the reasoning and methods (descriptive and mathematical) are clear and easy to follow.

Critical weaknesses

Ablation - I really like the ablation but felt it lacked some depth. As I understand it we are looking at the impact of (mostly) removing / changing a single component at a time? Because of this it’s hard to read the difference between e.g. full -> w/o VDAA vs full -> w/o motif. The relative difference is not that much on BBBP but pretty big on ClinTox. Generally this means it’s hard to reason about how much the difference comes from the change in the method, vs parameters, and how it works in different contexts. One way I really like to see this is to start combining more of the ablations. To consider the simplest version and add both individual parts and multiple parts together. Or the same removing parts from the full model. Given the complexity of the architecture presented it would be of great value for the reader to be able to parse why these parts in this combination, and then how to take that forward to future work.

Architecture diagram + explanation - having said that the paper and methods were clear, the architecture diagram could do with some improvements in my opinion. The VDAA is shown entering in two locations, but it’s unclear exactly how the inputs work there, and in the diagram, the floating boxes need more interpretation as to where everything links than would be ideal. The connected parts make sense, but I feel that improving this will significantly improve the reach of the paper.

The explanations of the background could be tightened, for instance the equations 1 and 2 feel overly verbose, as do 5-7. These are basically laying out the Transformer in graph context, then repeating the general attention mechanism. You could cut these and get back quite a lot of space to use to fill out details in the results / ablations sections.  

The transfer learning results seem mixed - which given that this additional information provided to the model ought to allow much richer representations does feel surprising. Either the base model has not significantly improved on the general models - might be nice to see a comparison on the Zinc pre-training task to a baseline ? - or the applications to small task specific datasets is not consistently helped by the additional information? A deeper analysis on what the real pattern is here would be very much appreciated.

Questions for authors

As above, the two key things I’d like to see the authors look at are some of the presentation of the architecture and the basic equations, as well as expanding the ablation section more extensively to understand the contribution of each different architectural component.

In equation 9 was there supposed to be a repeated \hat{G} ? I couldn’t see from the description there reference to the repeated inputs so I think this is a bug? Or if not needs some clarification.

5 comments

Q1:In equation 9 was there supposed to be a repeated G^\hat{G}? I couldn't see from the description there reference to the repeated inputs so I think this is a bug? Or if not needs some clarification. A1:Thank you for pointing this out. The repetition in Equation 9 is intentional rather than a typo, though we agree the notation could be clearer. The equation uses the standard attention form Att(Q,K,V)\mathrm{Att}(Q, K, V): the query QQ is the global graph vector g^\hat{g}, and the same motif matrix G^\hat{G} serves as both keys and values, which is the conventional way of writing cross-attention when keys and values share the same source. To eliminate the ambiguity we will rewrite the equation as Attn(g^,G^WK,G^WV)\mathrm{Attn}(\hat{g}, \hat{G}W_K, \hat{G}W_V) and add a sentence noting that the motif matrix G^\hat{G} is projected by independent, separately learned WKW_K and WVW_V (as in a standard multi-head attention implementation).

We thank the reviewer for this suggestion. Leave-one-out ablations are indeed difficult to compare across components, so we conducted an additive ablation: starting from the base model, VDAA and the motif branch are added individually and in combination, followed by GCMA. Each variant is pretrained independently from scratch under the same setting (ZINC 250K) and evaluated under the same downstream protocol.

Datasetbase+VDAA+motif+motif+GCMA+VDAA+motiffull
BBBP63.8±3.168.2±0.768.0±1.771.5±2.371.6±0.774.7±0.4
ClinTox77.6±2.086.9±1.681.0±1.985.1±1.891.5±1.494.4±0.9
BACE68.3±2.779.8±0.981.4±1.181.6±0.982.8±0.887.0±1.7
Tox2172.0±0.273.6±0.474.0±0.374.2±0.774.5±1.077.6±0.1
SIDER52.1±3.353.6±3.354.1±3.655.2±2.556.7±1.764.8±0.2

Several observations follow. VDAA yields consistent improvements regardless of the variant it is added to: on ClinTox, it improves the base model from 77.6 to 86.9 and the motif-only variant from 81.0 to 91.5. GCMA provides limited gains on its own but substantial gains once VDAA and the motif branch are present (91.5→94.4 on ClinTox, 82.8→87.0 on BACE, and 56.7→64.8 on SIDER). No individual component approaches the performance of the full model, which supports the chosen combination.

The dataset dependence noted by the reviewer is also visible in this table. On BBBP, VDAA and the motif branch yield comparable gains (63.8→68.2 and 63.8→68.0), whereas on ClinTox the gain from VDAA is substantially larger than that from the motif branch alone (77.6→86.9 versus 77.6→81.0). The components therefore play different roles on different datasets rather than contributing uniformly across tasks.

We thank the reviewer for pointing out the need for a more controlled analysis of transfer performance. The original comparison involves methods pretrained with different corpora, objectives, and training configurations. Therefore, the mixed results across published baselines do not, by themselves, establish whether the additional structural information is beneficial.

To address this issue, we conducted two additional controlled experiments. First, we performed an additive ablation in which every variant was pretrained independently from scratch under the same ZINC data, penalized-logP objective, and training budget. Both VDAA and the motif branch improve over the base model on all five evaluated benchmarks. GCMA provides further improvements in both the motif-only and VDAA+motif configurations, and the full model achieves the best performance on every benchmark. These results provide direct evidence that the proposed components contribute positively to transfer when each architecture is pretrained as a whole.

Second, we retrained SimSGT under the same 250K-molecule ZINC setting and downstream evaluation protocol as MoDANet. The results are summarized below:

DatasetMoDANetSimSGT (controlled)
BBBP74.7±0.471.4±3.8
ClinTox94.4±0.978.0±1.5
MUV77.0±0.976.9±0.4
HIV79.3±0.877.5±0.9
BACE87.0±1.781.2±0.3
Tox2177.6±0.176.4±0.8
SIDER64.8±0.262.3±1.5

Under this controlled setting, MoDANet achieves higher mean performance than SimSGT on all seven benchmarks. This controlled experiment shows that the additional structural information can be effectively transferred to downstream tasks.

We have added these experiments to Appendix F and revised the corresponding discussion to distinguish the original cross-method comparison from the new controlled evidence. Together, the independently pretrained ablation and the controlled SimSGT comparison support the effectiveness of the proposed architecture in learning transferable molecular representations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.