- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether glycan representations capture structural information beyond monosaccharide composition. It introduces GlycanStructBench, which combines fifteen prediction tasks with several data-splitting strategies and controlled structural diagnostics. The paper also presents GlycoTransportFormer, a model that uses linkage-conditioned transformations along rooted paths.
The main contribution is a benchmark for examining what glycan models learn and how their performance changes under different generalization settings. The experiments support the usefulness of linkage information and rooted-path encoding, while also identifying limitations of existing implementations and biological annotation tasks.
Critical strengths
-
The evaluation problem is well motivated. Glycans with the same composition can differ in connectivity, linkage configuration, and branching. Standard prediction scores do not reveal whether models preserve these distinctions. The combination of composition-disjoint evaluation, motif-cluster splits, exact-isomorphism splits, and controlled edits gives a more informative picture of model behavior. The benchmark should be useful for researchers developing glycan representations.
-
The central proposation has convincing experimental support. The ablations consistently show that removing linkage information or path transport degrade performance on subtype, type, and fine-grained taxonomy tasks. The subtype gains over GTF also persist under both composition-disjoint and exact-isomorphism-disjoint evaluation. These results support the usefulness of linkage information and rooted-path encoding across multiple tasks and evaluation settings.
-
The diagnostic experiments provide useful findings beyond the proposed model’s performance. The edit tests identify structural distinctions that some implementations fail to preserve, as well as sensitivity to node ordering. The retrieval experiments show that structural nearest neighbours can outperform the learned model on the sparse disease and tissue tasks. These findings help explain what the benchmark scores can support and provide practical guidance for future model comparisons. I appreciate that the paper includes these results and accounts for them in its conclusions.
Critical weaknesses
The benchmark's novelty is mainly in the evaluation framework. Several task families come from existing resources, and GlycanML already uses motif-based clustering to evaluate structural generalization. The contribution here lies in bringing complementary split strategies, identity checks, and representation diagnostics into a common evaluation. This is a useful extension, although its originality rests on the additional questions these evaluations answer and the findings they enable. The breadth of the task suite alone does not constitute a major advance.
Questions for authors
Could the authors extend the composition-only baseline in Table 26 to glycan subtype prediction and a taxonomy task such as species classification, under the composition-disjoint split? This would help assess how much predictive performance can be achieved using residue counts alone.