- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This study proposes an Explainable Cross-Modal Learning Framework (ECMLF) for cold-start protein–protein interaction (PPI) prediction. ECMLF integrates amino acid sequence information with four external biomedical knowledge graphs to model similarities between novel and known proteins. An adaptive gated fusion module jointly learns protein similarity and PPI prediction, while a dual-pathway explanation mechanism provides biologically grounded insights into predicted interactions. Experiments on two real-world datasets demonstrate improved predictive performance and explanation quality over existing methods.
Critical strengths
-
This study proposes ECMLF, which enriches the contextual information available for protein–protein interaction (PPI) prediction by incorporating four biologically meaningful graph attention networks alongside protein sequence representations. This multimodal design constitutes a valuable methodological contribution.
-
Beyond its technical contributions, this research work examines the model’s attention scores for protein sequence and graph representations. This interpretability analysis offers potential insight into the mechanisms underlying its PPI predictions and strengthens the presentation of the work.
-
ECMLF demonstrates strong performance across extensive experimental evaluations, as measured by accuracy (ACC), recall (REC), precision (PRE), F1 score, area under the ROC curve (AUC), area under the precision–recall curve (AUPR), and Matthews correlation coefficient (MCC). Together, these metrics provide broad evidence of the constructed model’s predictive advantages.
Critical weaknesses
-
ECMLF’s performance gains rely substantially on the availability and completeness of four types of external biomedical associations. However, the paper does not systematically evaluate the model under missing modalities, sparse annotations, or distribution shifts. Consequently, its robustness to poorly annotated proteins and incomplete knowledge graphs in real-world scenarios remains unclear.
-
Considering the paper as a whole, ECMLF introduces additional multimodal information from knowledge graphs to help the model better capture the biological mechanisms of proteins. Subsequently, the subsequent gated fusion mechanism combines representations from four knowledge graphs and a protein sequence encoder before passing them to the PPI prediction head. The pipeline in stage II does not appear to differ substantially from conventional approaches or offer a clear algorithmic innovation. Could the performance gains over the baselines models therefore be attributed primarily to the additional multimodal information?
-
The included interpretability analyses of individual examples can shed light on plausible mechanisms underlying the model’s correct predictions and thus complement the main findings. However, they do not constitute a large-scale statistical analysis or provide robust evidence for those mechanisms. (minor)
Questions for authors
I recommend that the authors further clarify the core algorithmic innovations underlying the proposed method’s performance gains and provide additional evidence of the generalizability of these innovations in real-world application scenarios.
4 comments
The following summarizes our additional experiments. Numerical results available in the manuscript tables are reproduced below; for other experiments, we refer to the corresponding sections of the revised manuscript.
W1(1):Modality Coverage Ablation. (see revised manuscript, Appendix A.6 "Modality Coverage Ablation")
SCS
| Coverage | Modality | ACC | REC | PRE | F1 | AUC | AUPR | MCC | F-rank |
|---|---|---|---|---|---|---|---|---|---|
| 1 | M1 | 60.91 | 87.65 | 58.77 | 64.98 | 74.88 | 66.60 | 37.86 | 11.14 |
| 1 | M2 | 61.75 | 87.28 | 58.84 | 65.24 | 73.48 | 65.06 | 39.02 | 8.43 |
| 1 | M3 | 60.99 | 87.57 | 58.67 | 64.99 | 75.92 | 68.19 | 37.59 | 10.29 |
| 1 | M4 | 61.03 | 87.24 | 58.87 | 64.93 | 71.94 | 63.61 | 37.58 | 12.14 |
| 2 | M1+M2 | 61.46 | 88.44 | 58.60 | 65.45 | 75.16 | 66.80 | 39.56 | 7.00 |
| 2 | M1+M3 | 60.92 | 88.45 | 58.49 | 65.24 | 77.41 | 69.73 | 38.16 | 8.14 |
| 2 | M1+M4 | 60.95 | 88.12 | 58.72 | 65.18 | 74.35 | 67.15 | 38.08 | 9.86 |
| 2 | M2+M3 | 61.59 | 88.08 | 58.47 | 65.40 | 76.49 | 68.62 | 38.93 | 7.00 |
| 2 | M2+M4 | 61.77 | 87.80 | 58.78 | 65.46 | 72.47 | 63.99 | 39.18 | 7.57 |
| 2 | M3+M4 | 60.98 | 88.01 | 58.51 | 65.13 | 76.03 | 68.57 | 37.63 | 9.71 |
| 3 | M1+M2+M3 | 61.32 | 89.19 | 58.22 | 65.58 | 77.68 | 69.93 | 39.61 | 4.86 |
| 3 | M1+M2+M4 | 61.45 | 88.89 | 58.51 | 65.61 | 74.60 | 67.34 | 39.69 | 5.86 |
| 3 | M1+M3+M4 | 60.91 | 88.84 | 58.34 | 65.35 | 77.53 | 70.12 | 38.22 | 7.57 |
| 3 | M2+M3+M4 | 61.56 | 88.49 | 58.31 | 65.51 | 76.51 | 68.82 | 38.92 | 6.29 |
| 4 | M1+M2+M3+M4 | 61.30 | 89.63 | 58.01 | 65.69 | 77.75 | 70.23 | 39.63 | 4.14 |
FCS
| Coverage | Modality | ACC | REC | PRE | F1 | AUC | AUPR | MCC | F-rank |
|---|---|---|---|---|---|---|---|---|---|
| 1 | M1 | 72.99 | 77.74 | 62.87 | 66.26 | 85.79 | 76.41 | 49.52 | 11.57 |
| 1 | M2 | 78.49 | 72.41 | 68.21 | 68.98 | 86.02 | 76.32 | 53.86 | 7.86 |
| 1 | M3 | 73.85 | 78.02 | 63.10 | 67.03 | 85.60 | 75.69 | 50.35 | 11.00 |
| 1 | M4 | 74.16 | 77.56 | 63.80 | 67.08 | 85.80 | 76.02 | 50.79 | 10.29 |
| 2 | M1+M2 | 77.85 | 75.62 | 66.78 | 69.30 | 86.48 | 77.38 | 54.21 | 5.71 |
| 2 | M1+M3 | 72.14 | 79.26 | 61.44 | 66.20 | 86.08 | 76.74 | 48.83 | 11.43 |
| 2 | M1+M4 | 72.37 | 78.80 | 62.08 | 66.21 | 86.31 | 77.15 | 49.18 | 10.29 |
| 2 | M2+M3 | 78.13 | 75.47 | 66.84 | 69.59 | 86.24 | 76.62 | 54.31 | 6.00 |
| 2 | M2+M4 | 78.24 | 74.52 | 67.38 | 69.42 | 86.34 | 76.87 | 54.22 | 5.86 |
| 2 | M3+M4 | 73.16 | 79.23 | 62.25 | 66.96 | 85.98 | 76.27 | 49.97 | 10.57 |
| 3 | M1+M2+M3 | 77.18 | 78.30 | 65.30 | 69.61 | 86.71 | 77.68 | 54.19 | 4.57 |
| 3 | M1+M2+M4 | 77.38 | 77.29 | 65.85 | 69.47 | 86.85 | 78.00 | 54.14 | 5.14 |
| 3 | M1+M3+M4 | 71.72 | 80.64 | 60.85 | 66.44 | 86.49 | 77.40 | 48.89 | 9.29 |
| 3 | M2+M3+M4 | 77.69 | 76.89 | 65.91 | 69.68 | 86.46 | 77.06 | 54.14 | 5.86 |
| 4 | M1+M2+M3+M4 | 76.54 | 79.45 | 64.23 | 69.49 | 86.97 | 78.19 | 53.65 | 4.57 |
W1(2):Graph Association Ablation. (see revised manuscript, Appendix A.6 "Graph Association Ablation")
W2(2):Further Reconstruction Ablation. (see revised manuscript, Appendix A.6 "Further Reconstruction Ablation")
W2(3):Bridge Entity Identification Ablation. (see revised manuscript, Appendix A.7 "Bridge Entity Identification Ablation")
I thank the authors for their response. After the discussion, I understand that the incorporated bridge reconstruction mechanism constitutes the paper’s primary methodological innovation. After holistic consideration, I decide to maintain my original rating score.
Thank you for your understanding. We appreciate your time and consideration.
W1: We acknowledge that ECMLF’s performance depends on the availability of the four types of external biomedical associations. To further assess their impact, we conducted two additional experiments. (1) To assess how the number of available graph modalities affects prediction performance for new proteins, we conducted a modality coverage experiment. Specifically, unlike the modality ablation experiment in Table 3 of the original manuscript, which trains a separate model for each modality combination, this experiment uses the full model trained with all four graph modalities. During the testing phase, we selected new proteins covered by all four graph modalities as the test set and masked their graph modality information to simulate new proteins with modality coverage of n (n=4,3,2,1). The results (see revised manuscript, Appendix A.6 "Modality Coverage Ablation") indicate that broader graph modality coverage generally improves prediction performance for new proteins. However, even when only one graph modality is available, the model retains a degree of predictive ability for new proteins. (2) To assess whether the model’s advantage persists with few biomedical graph associations, we conducted a graph association ablation experiment comparing the full model with its sequence-only variant. Specifically, we progressively removed 25%, 50%, 75%, and 100% of each new protein’s biomedical graph associations and reported performance stratified by the amount of graph information remaining. Based on the experimental results, as graph associations decrease, the model’s performance gradually declines. But it generally outperforms the sequence-only variant, even for new proteins with few or no remaining graph associations (see revised manuscript, Appendix A.6 "Graph Association Ablation").
W2: (1) We acknowledge that Stage II does not appear to differ substantially from conventional approaches. However, in Stage I, we introduce bridge reconstruction as an auxiliary supervision task to learn latent similarities between known and new proteins, enabling interaction patterns learned from known proteins to transfer to new proteins. This specific bridge reconstruction objective is the core design of our model and is absent from existing conventional PPI methods like KGF-GNN and HEENN. Specifically, bridge reconstruction is designed to address the absence of PPI records for new proteins by learning similarities through biomedical associations. (2) The reconstruction ablation experiment has already demonstrated the importance of this component for cold-start PPI prediction (see revised manuscript, Appendix A.6 "Reconstruction Ablation"). To exclude the influence of sequence information and further assess the contribution of bridge reconstruction, we conducted an additional reconstruction ablation experiment without the sequence branch. The results further support the contribution of bridge reconstruction to cold-start prediction (see revised manuscript, Appendix A.6 "Further Reconstruction Ablation"). (3) Beyond the performance changes shown by the reconstruction ablation above, we have added another experiment to further examine how bridge reconstruction mechanistically addresses the absence of PPI records for new proteins. In the Graph-Pathway Explainability section of the original manuscript, we evaluated the model’s ability to capture key bridge entities that connect new proteins to known proteins sharing similar interaction patterns, thereby enabling the model to transfer interaction patterns learned from known proteins to new proteins. For new proteins lacking PPI records, our bridge reconstruction task provides auxiliary supervision, guiding the model to capture these key bridge entities to uncover latent associations between new and known proteins. Therefore, we compared the full model with its variant without bridge reconstruction in their ability to capture these key entities. The results also show that removing bridge reconstruction reduces the model’s ability to identify key entities connecting known and new proteins (see revised manuscript, Appendix A.7 "Bridge Entity Identification Ablation").
W3: In Section 5.4, both sequence-pathway and graph-pathway explanations are evaluated for every protein pair in the full test set, providing large-scale quantitative evidence of their biological relevance. The case study in Section 5.4 is presented to specifically illustrate sequence-pathway explainability for a selected protein pair, offering biological evidence for a plausible mechanism underlying the correct prediction. However, due to page limits, we did not conduct case study for all protein pairs and the existing case study already supports the sequence-pathway explanation analysis. The case study analyses could be extended to all protein pairs, but this would require substantial time and computational resources.