acceptodds
Under review as a conference paper at ICLR 2027

GlycanStructBench: Benchmarking Structure-Faithful Glycan Representations with Linkage-Aware Rooted Path Transport

12 views

Abstract

Glycans pose a distinctive challenge for representation learning because their functions arise from branched architectures, not residue composition alone. Identical residue counts can differ in linkage, stereochemistry, root orientation, and terminal organization, so composition does not determine structure. This raises a central question: do models capture structural information or exploit compositional shortcuts? We introduce GlycanStructBench, a normalized-key-disjoint benchmark for evaluating whether glycan representations preserve structural distinctions across fifteen structural, taxonomic, and biological tasks. Its identity unit is the normalized structure key, not chemical identity. The benchmark includes composition-disjoint and motif-profile/cluster-disjoint stress-test splits, which withhold residue-count groups and whole clusters, respectively, and controlled probes of topology, linkage, and root-dependent processing. We also evaluate GlycoTransportFormer as a representative rooted-tree encoder with linkage-conditioned common-root-frame transport, not as a claim that this operator is optimal. Across seven models and five seeds, the encoder leads the structural and taxonomy families on the normalized-key and composition-disjoint partitions, with paired gains of 1.28 and 8.71 points, and 0.91 and 4.15 points under composition shift; all five paired seed-level differences are positive. On the motif-profile/cluster-disjoint partition it leads on subtype, type, and all eight taxonomy ranks. We report family-level paired effects and cross-partition consistency rather than a single mean rank. A parameter-free retrieval baseline exceeds the encoder on the two sparse biological audit tasks. These results show why glycan benchmarks should control for composition when assessing structure-faithful representations.

Primary area:
datasets and benchmarks
Keywords:
Glycan structure representations, Structure-disjoint benchmarking, Linkage-aware tree representation learning, Composition-disjoint evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HjwnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether glycan representations capture structural information beyond monosaccharide composition. It introduces GlycanStructBench, which combines fifteen prediction tasks with several data-splitting strategies and controlled structural diagnostics. The paper also presents GlycoTransportFormer, a model that uses linkage-conditioned transformations along rooted paths.

The main contribution is a benchmark for examining what glycan models learn and how their performance changes under different generalization settings. The experiments support the usefulness of linkage information and rooted-path encoding, while also identifying limitations of existing implementations and biological annotation tasks.

Critical strengths
  1. The evaluation problem is well motivated. Glycans with the same composition can differ in connectivity, linkage configuration, and branching. Standard prediction scores do not reveal whether models preserve these distinctions. The combination of composition-disjoint evaluation, motif-cluster splits, exact-isomorphism splits, and controlled edits gives a more informative picture of model behavior. The benchmark should be useful for researchers developing glycan representations.

  2. The central proposation has convincing experimental support. The ablations consistently show that removing linkage information or path transport degrade performance on subtype, type, and fine-grained taxonomy tasks. The subtype gains over GTF also persist under both composition-disjoint and exact-isomorphism-disjoint evaluation. These results support the usefulness of linkage information and rooted-path encoding across multiple tasks and evaluation settings.

  3. The diagnostic experiments provide useful findings beyond the proposed model’s performance. The edit tests identify structural distinctions that some implementations fail to preserve, as well as sensitivity to node ordering. The retrieval experiments show that structural nearest neighbours can outperform the learned model on the sparse disease and tissue tasks. These findings help explain what the benchmark scores can support and provide practical guidance for future model comparisons. I appreciate that the paper includes these results and accounts for them in its conclusions.

Critical weaknesses

The benchmark's novelty is mainly in the evaluation framework. Several task families come from existing resources, and GlycanML already uses motif-based clustering to evaluate structural generalization. The contribution here lies in bringing complementary split strategies, identity checks, and representation diagnostics into a common evaluation. This is a useful extension, although its originality rests on the additional questions these evaluations answer and the findings they enable. The breadth of the task suite alone does not constitute a major advance.

Questions for authors

Could the authors extend the composition-only baseline in Table 26 to glycan subtype prediction and a taxonomy task such as species classification, under the composition-disjoint split? This would help assess how much predictive performance can be achieved using residue counts alone.

Reviewer_mx5wOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This is a solid paper proposing a new glycan benchmark GlycanStructBench that partitions the glycan data according to structure. The paper argues that current representation approaches collapses sugars that are structurally different. The new benchmark also introduces different partitions of the data for improved measuring of generalisation. The paper also as secondary point introduces a tailores Transformer architecture GlycoTransformer and benchmarks it against existing tools. The validation and dataset construction is thorough,

Critical strengths

New benchmark that addresses previous limitations. Good baselines.

Critical weaknesses

None really. Only fundamental problem is that we the size of the dataset is not very large reflecting that there are a limited number of well-charaterised glycans.

Questions for authors

Why did you remove line spacing between paragraphs? ;-)

Reviewer_Jg2KOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, the authors introduce GlycanStructBench to better understand how well machine learning models learn glycan structures. The benchmark includes 15 prediction tasks related to glycan structure, taxonomy, and biological properties. To make the evaluation more realistic, the authors use different data-splitting strategies that test whether models can handle unfamiliar glycan compositions and structures without relying on information leaked from the training data. They also develop GlycoTransportFormer, a graph-based model that considers glycosidic linkages and paths from the root of a glycan. By combining a transport mechanism with attention, the model learns how different branches contribute to the overall glycan structure.

Critical strengths
  1. The paper has several evaluation regimes: composition shift, cluster shift, exact graph equivalence and class-safe cluster.
  2. It provides rich ablation study, where authors report also negative results.
  3. It includes broad task coverage.
Critical weaknesses
  1. Limited evidence for the proposed noncommutative orthogonal transport mechanism (see Q1).
  2. GlycoTransportFormer has approximately 6 million parameters, while some baseline models have fewer than 100,000. I doubt the great number of parameters is an advantage to GlycoTransportFormer (see Q2).
  3. Parameter-free nearest-neighbor baseline performs better than GlycoTransportFormer on disease association and tissue localization under the cluster-disjoint split. This shows that GlycoTransportFormer struggles on biological prediction tasks.
Questions for authors
  1. Since the commutative and additive variants achieve comparable performance, can the authors justify why they used noncommutative orthogonal transport mechanism?
  2. Can the authors compare with a model that has almost the same number of parameters?
  3. Are the performance improvements large enough to justify the additional training time and computational cost?
  4. Would the model achieve similar results if different groups of glycans were used for training and testing?
  5. Could the authors directly test whether the model can distinguish glycans that have the same composition but different linkages?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.