- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors introduce PlantMetBench, a multimodal knowledge graph benchmark for predicting enzyme–reaction associations in plant metabolism, with a focus on organism-disjoint link prediction. Built upon PlantMetWiki, the benchmark integrates proteins, genes, metabolites, and biochemical reactions, providing multimodal node features, reproducible dataset splits, and evaluation protocols. The authors evaluate heterogeneous GNN architectures against non-graph baselines, showing that relational information can improve enzyme–reaction prediction. Additional experiments investigate generalization across species and reactions, structural shortcuts, and pathway reconstruction. The results highlight both the potential of graph-based approaches and the remaining challenges in biosynthesis prediction
Critical strengths
- PlantMetBench addresses a gap in biosynthesis prediction by providing a standardized multimodal knowledge graph benchmark. The integration of biological entities, domain-specific node features, reproducible data splits, and evaluation protocols makes it a potentially valuable resource for the computational biology and graph learning communities
- The paper provides a comprehensive comparison of several GNN architectures and non-graph baselines. Beyond standard performance comparisons, the authors consider the evaluations also on different data-splitting strategies and includes ablation studies investigating the impact of graph structure and potential structural shortcuts
Critical weaknesses
- A major challenge in molecular machine learning is designing dataset splits that reliably assess generalization beyond the training data. Although PlantMetBench adopts an organism-disjoint split, the substantial performance drop on unseen reactions highlights the importance of distinguishing cross-species transfer from generalization to novel reactions. While the authors acknowledge this limitation, the strong dependence on previously observed reaction information raises concerns about the extent to which the default evaluation captures broader generalization capabilities
- A substantial number of nodes in the dataset lack feature representations and are instead assigned zero vectors. This may affect the performance of the evaluated architectures, making it difficult to determine whether the reported results reflect the inherent complexity of the task or the limited availability of input information
Questions for authors
- Considering the substantial performance drop on unseen reactions, why is the organism-disjoint split preferred as the default evaluation setting? Would an unseen-reaction split provide a more comprehensive assessment of generalization to novel enzyme–reaction associations?
- Due to the use of zero vectors to represent missing node features, how does this choice affect the performance of the evaluated architectures? It would be interesting to distinguish limitations arising from missing input information from those related to the inherent difficulty of the prediction task
Minor comments
- The paper includes GAT among the heterogeneous GNN baselines, although the original GAT architecture was designed for homogeneous graphs. A brief clarification in the main text on how GAT is adapted to handle different node and edge types would be helpful
- Some tables, including Tables 1, 2, and 3, contain vertical lines, which could be removed to improve readability