acceptodds
Under review as a conference paper at ICLR 2027

GraphBTE: Graph Byte Triple Encoding

Abstract

Molecular property prediction commonly uses graph-based GNNs or SMILES-based Transformers, where SMILES tokenization captures molecular substructures beyond individual atoms. However, BPE-based SMILES tokenization has two problems regarding invariance under SMILES enumeration: SMILES-enumeration dependence and inconsistent subgraph tokenization. Direct tokenization of molecular graphs addresses these limitations but leads to two additional problems: connectivity uncertainty in pair-based merging and loss of atom-level positions. To address these limitations, we propose Graph Byte Triple Encoding (GraphBTE), which represents each merge rule using two adjacent tokens and the canonical representation of their merged subgraph, while incorporating atom-level positional information through spectral positional encoding (SPE). Across seven MoleculeNet classification datasets, GraphBTE outperforms GraphBPE with a GNN and achieves higher average ROC-AUC than SMILES BPE under the same Transformer architecture. Further analyses show that GraphBTE is invariant to SMILES enumeration, consistently tokenizes shared subgraphs in the analyzed examples, and distinguishes different merged subgraphs formed from the same token pair. SPE further improves prediction performance, and the best-performing vocabulary size is strongly associated with molecular size.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Molecular Graph Tokenization, Molecular Representation Learning, Molecular Property Prediction, Substructure Vocabulary Learning, Spectral Positional Encoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3K1gOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposed GRAPH BYTE TRIPLE ENCODING, where the merge rule is based on two adjacent tokens and the canonical representation of their merged subgraph, to address the enumeration dependence and inconsistent subgraph tokenization issue of SMILES, and connectivity uncertainty in pair-based merging.

Critical strengths
  1. The motivation is very clear, the proposed method is reasonable to reduce the ambiguity of a graph representation compare to baseline methods.
  2. The paper is well organized, the figures and examples are illustrative, the experiment set-up and results looks reasonable and promising.
Critical weaknesses
  1. The inconsistent subgraph tokenization is not fully resolved, because the order of nodes is not deterministic so that tokenization results could be different.
    For example, Figure 2 Example M2 could have 2 different orders, given vocab (C, O, N, (C, O, C-O)), COCN is tokenized to (C, O, C-O)-C-N while NCOC corresponds to N-(C, O, C-O)-C. In Algorithm 1 Line 7: for adjacent token pairs (a,b) in G; Algorithm 2 Line 4: for adjacent tokens in G_token, the order of G is not described.
  2. Structural Ambiguity still exists: although the original atom connection is preserved during the merging to construct the canonical subgraph, once the merge is done, the connection information of 2 subgraph is still missing in the final representation. e.g. in N-(C, O, C-O)-C, if N connects to C or O is not represented.
  3. SPE plays an important role in improving the metrics, the author should also compare with the GraphBPE+SPE to make fair comparison.
Questions for authors

NA

Reviewer_5Km1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes GraphBTE, a data-driven tokenizer for molecular graphs aimed at molecular property prediction. It argues that SMILES BPE depends on the chosen SMILES string and tokenizes the same subgraph inconsistently across molecules, and that pair-based graph BPE cannot tell apart different subgraphs formed by the same token pair and loses atom-level positions once atoms are merged. GraphBTE keys each merge rule on the two adjacent tokens together with the canonical SMILES of their merged subgraph, and adds a spectral positional encoding computed on the atom graph and summed within each token. Validation uses seven MoleculeNet classification datasets with a one-layer GIN (against GraphBPE, atom-level, and rule-based fragmentation tokenizers) and a four-layer Transformer (against ChemBERTa's SMILES BPE), plus a randomized-SMILES invariance test, an ablation of the positional encoding, a count of ambiguous token pairs, and a post hoc analysis of vocabulary size. Experimental results show that GraphBTE outperforms the selected baselines on most datasets.

Critical strengths
  1. A simple tokenizer design for a relevant problem. Adding the canonical SMILES of the merged subgraph to each merge rule lets a learned graph vocabulary separate subgraphs that a pair-only rule conflates.

  2. Transparent reporting. Validation-selected vocabulary and depth, and appendix tables on the positional encoding, depth, and vocabulary size make the results checkable.

Critical weaknesses
  1. The controls and baselines are inadequate. Atom and GraphBPE are never run with the tokenizer-independent positional encoding, GraphBPE loses its published components, the one-layer backbone is below a standard GIN, and the closest published tokenizers are missing.

  2. The method and the results do not support the motivation. The paper's own ablations credit most of the gain to the encoding rather than to the learned merges, the summed encoding does not say where a substituent sits, and the token graph keeps no attachment atoms between tokens.

Questions for authors
  1. Section 5.1 credits Table 1 to the learned tokens (lines 334 to 335), but SPE does not depend on the tokenizer, and Atom and GraphBPE are never run with it; without SPE, GraphBTE leads GraphBPE only 69.2 to 68.0 (Table 2). What do Atom + SPE and GraphBPE + SPE score?
  2. GraphBPE is run without the ring contraction and contextualizer of its own experiments (lines 122 to 124), which leaves plain label-pair merging. How does it perform in its published configuration?
  3. The closest published methods, principal subgraph extraction [1], connection-aware motif mining [2], node pair encoding [3], and graph BPE for Transformers [4], are neither cited nor compared. How does GraphBTE compare with them?
  4. The one-layer GIN is weak: Atom falls below the non-pretrained GIN of Hu et al. [5] on Tox21, ToxCast, MUV, and HIV (66.8 vs 71.8 on MUV). Does the gain hold on a standard five-layer GIN or GraphGPS [6]?
  5. V18 has no merges, so it is Atom + SPE, yet it is GraphBTE's best vocabulary on Tox21 and ToxCast (Table 7), and Figure 8 already starts above 70.0 at V18 against 70.9 at V38. How much do the learned merges add?
  6. SPE is meant to restore the atom positions lost by merging (lines 088 to 091), but Eq. 1 sums the atom encodings within a token, losing which atom carries a bond, and SPE helps Atom, where nothing is merged, more than GraphBTE (Figure 8, Table 2). Does SPE separate the ortho, meta, and para isomers of Figure 1 (c)?
  7. The token graph keeps only token-to-token edges (Algorithm 2, step 7), and 61.59% to 83.55% of adjacent token pairs in GraphBTE's own V38 graphs are ambiguous (Figure 5). Do the edges carry bond types or attachment atoms, and if not, how does the model separate these cases?
  8. The vocabulary token is c_ab alone (Algorithm 1), and c_ab already separates bond types and attachment sites (lines 151 to 153). What do a and b add, and how does the triple compare with a c_ab-only key as in [1]?

[1] Xiangzhe Kong, Wenbing Huang, Zhixing Tan, and Yang Liu. Molecule generation by principal subgraph mining and assembling. In NeurIPS, 2022.

[2] Zijie Geng, Shufang Xie, Yingce Xia, Lijun Wu, Tao Qin, Jie Wang, Yongdong Zhang, Feng Wu, and Tie-Yan Liu. De novo molecular generation via connection-aware motif mining. In ICLR, 2023.

[3] Gang Liu, Jie Chen, Yihan Zhu, Michael Sun, Tengfei Luo, Nitesh V. Chawla, and Meng Jiang. Graph diffusion transformers are in-context molecular designers. In ICLR, 2026.

[4] Zeyuan Guo, Enmao Diao, Cheng Yang, and Chuan Shi. Graph tokenization for bridging graphs and transformers. In ICLR, 2026.

[5] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In ICLR, 2020.

[6] Ladislav Rampášek, Mikhail Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer. In NeurIPS, 2022.

Reviewer_jLquOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Graph Byte Triple Encoding (GraphBTE), a molecular graph tokenization method designed to address limitations of SMILES-based and pair-based graph tokenization. Specifically, GraphBTE extends conventional pair-based merging to triple-based merging by incorporating the canonical representation of the merged subgraph, thereby distinguishing different bond types and attachment sites associated with the same token pair. The method further integrates spectral positional encoding (SPE) computed from the original atom-level graph to preserve structural information during tokenization. Experiments on seven MoleculeNet classification datasets demonstrate improved average ROC-AUC over GraphBPE with a GNN backbone and SMILES BPE under a shared Transformer architecture.

Critical strengths
  1. The paper identifies meaningful limitations of existing molecular tokenization methods and provides a clear motivation for incorporating connectivity information into graph-based merging.

  2. The proposed triple-based merging strategy is conceptually simple and offers an intuitive way to distinguish molecular substructures with different bond types or attachment sites.

  3. The evaluation covers seven molecular property prediction benchmarks and includes extensive analyses of structural ambiguity, positional encoding, vocabulary size, and SMILES enumeration invariance.

Critical weaknesses
  1. GraphBTE primarily extends GraphBPE [1] by incorporating the canonical representation of the merged subgraph into an otherwise similar frequency-based merging procedure. While this modification addresses a meaningful ambiguity, the resulting method remains closely related to existing graph-based tokenization strategies, particularly MiCaM [2], which has already explored connectivity-aware molecular subgraph merging. Moreover, the integration of spectral positional encoding relies on an established technique rather than introducing a fundamentally new mechanism. The paper would benefit from a more precise characterization of its methodological contribution beyond combining topology-aware merging with existing positional encoding.

  2. In Table 1, GraphBTE uses SPE whereas GraphBPE does not, making it difficult to attribute the reported improvement solely to the proposed tokenization strategy. Table 2 shows that removing SPE reduces GraphBTE's average ROC-AUC from 70.90 to 69.18, leaving a smaller improvement over GraphBPE. Although SPE is evaluated with several fragmentation baselines in Appendix Table 9, the crucial GraphBPE+SPE comparison is missing. A controlled factorial ablation with identical architectures and vocabulary budgets is needed to establish the independent contributions of the two components.

  3. The triple-based merge rule ensures that a matched pair corresponds to a specific merged subgraph, but it does not necessarily guarantee consistent final tokenization of shared subgraphs under different molecular contexts, particularly when overlapping merges occur. The consistency analysis is primarily qualitative. Similarly, SPE aggregates atom-level positional encodings through summation, which does not guarantee preservation of all relative atomic positions. The paper would benefit from quantitative cross-molecule consistency measurements and controlled experiments involving structurally distinct molecular isomers, especially the ortho, meta, and para examples motivating the method.

  4. The main graph-based comparisons rely on a one-layer GIN, while the Transformer experiments use a relatively small four-layer architecture trained from scratch. Although Appendix Table 8 investigates GNN depth, the evaluation does not establish whether the proposed tokenization remains beneficial when integrated with stronger molecular encoders that explicitly model bond information. Furthermore, Table 3 compares tokenization strategies with different vocabulary sizes and positional encodings, making the source of the relatively modest Transformer improvement difficult to determine. Additional controlled experiments with stronger molecular architectures would substantially strengthen the practical relevance of the method.

  5. Figure 5 and Appendix Table 10 report that a substantial proportion of distinct adjacent token pairs correspond to multiple merged subgraphs. However, these statistics are calculated from the final token graphs produced by GraphBTE rather than directly measuring ambiguous merge decisions encountered by GraphBPE. Consequently, the reported ambiguity rates do not necessarily reflect the frequency or predictive impact of incorrect or insufficiently specific merges in the baseline. A direct comparison of merge behavior, structural distinctions preserved, and downstream performance on ambiguity-sensitive molecules would provide stronger evidence for the proposed design.

[1] Yuchen Shen and Barnabás Póczos. GraphBPE: Molecular Graphs Meet Byte-Pair Encoding. arXiv preprint arXiv:2407.19039, 2024.

[2] Zijie Geng et al. De Novo Molecular Generation via Connection-aware Motif Mining. International Conference on Learning Representations (ICLR), 2023.

Questions for authors

Please refer to the Weaknesses section.

Comments on the paper

1 comment

Dear reviewers,

The rebuttal period is ending soon. Please read the authors rebuttal and check if they addressed your concerns. Please acknowledge the rebuttal and adjust your scores accordingly.

Thanks.

-AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.