- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PPI2Text, a multimodal language model for generating free-form descriptions of protein-protein interactions from amino acid sequences. The method uses ESM3 to encode individual proteins, constructs a residue-level pair map to capture interactions between protein representations, and introduces PaCo-RoPE to align the positional information of the two proteins and their pairwise representations in the language decoder. The authors also construct PPI2Text-Dataset, containing 351K protein pairs with descriptions synthesized from curated biological evidence. Experiments on temporal holdout and C3-hard splits show improvements over the compared baselines in both text generation and evidence-based factuality metrics.
Critical strengths
-
The paper addresses a meaningful problem by extending conventional PPI classification to free-form interaction descriptions. The proposed architecture is well motivated, particularly the pairwise residue modeling and PaCo-RoPE, which go beyond simply combining individual protein embeddings.
-
The construction of a large-scale PPI description dataset is a valuable contribution. The integration of evidence from multiple biological databases, together with temporal holdout and C3-hard evaluations, provides a reasonably comprehensive experimental setup. The ablation studies also support the effectiveness of the proposed components.
Critical weaknesses
- The effectiveness of the pair-map architecture is not sufficiently isolated or validated. The paper does not examine whether the learned representations correspond to actual residue contacts or interaction interfaces. Moreover, there seems to be no baseline using the same ESM3 backbone that feeds the cross-attended 1D representations directly to the decoder without constructing the 2D pair map, making it difficult to attribute the improvements specifically to pair-map modeling and PaCo-RoPE.
- The reliability and practical applicability of the generated descriptions require further investigation. Although the pipeline uses evidence-tiered control tags to constrain the labeler LLM, the training data still relies on LLM-synthesized descriptions that may contain hallucinated biological details, while the factuality evaluation primarily uses an LLM judge validated on only 50 protein pairs. Additionally, the assumption that all input protein pairs interact limits the framework's applicability to real-world settings.
Questions for authors
-
Can the authors provide evidence that the learned pair-map representations correspond to actual residue-level interaction patterns? For example, could they evaluate the pair-map features against known protein interaction interfaces or contact maps?
-
How does PPI2Text behave when given a pair of proteins that do not interact? Does it still generate a confident interaction description, or can it express uncertainty about the existence of an interaction?
-
Could the authors provide a more extensive human expert evaluation of the generated descriptions, particularly regarding mechanistic correctness and hallucination?
5 comments
Author Response to Reviewer wjCj (2/3)
[CONTINUED]
W2.2 & Q3 (extended human evaluation)
Following the reviewer's suggestion, we conducted a claim-level human audit of the descriptions generated for 59 C3-hard pairs, cross-checking each claim against the raw evidence in a model-blinded way. We distinguished between two categories of claims: (1) sequence-inferable (SI) claims, which could in principle be inferred from protein sequences (e.g., family, domain, function, localization, and interaction type), and (2) non-sequence-inferable (nSI) claims, which involve literature-specific knowledge not directly recoverable from sequences alone (e.g., named complexes, diseases, drugs, and other specific proteins).
| C3-hard, 59 pairs | SI (wrong) ↓ | nSI (supported) ↑ | nSI (contradicted) ↓ | Mec.Hal. ↓ | Certainty vs. evidence (ρ) ↑ |
|---|---|---|---|---|---|
| Retrieval (ESM3) | 31% | 20% | 17% | 46% | 0.18 |
| Qwen3-32B, 3-shot, seq only | 59% | 2% | 27% | 85% | 0.09 |
| PPI2Text | 21% | 40% | 12% | 16% | 0.43 |
SI: sequence-inferable; nSI: non-sequence-inferable; Mec.Hal.: rate of unsupported mechanism claims, where the model asserts a mechanism absent from the source evidence.
PPI2Text consistently outperforms both baselines in factually grounded and uncertainty-aware generation. It achieves the lowest error rate on sequence-inferable claims (21% vs. 31% and 59%) while producing fewer hallucinations on literature-specific claims (12% vs. 17% and 27%). More importantly, its mechanism hallucination rate is only 16%, compared with 46% for retrieval and 85% for Qwen3-32B. Furthermore, the certainty expressed in its descriptions exhibits a substantially stronger correlation with the underlying evidence strength (ρ = 0.43 vs. 0.18 and 0.09), suggesting that PPI2Text better adjusts the confidence of its language to the available evidence rather than presenting uncertain hypotheses as established facts.
Nevertheless, hallucinations are not entirely eliminated: 12% of PPI2Text's literature-specific claims contradict the source evidence, corresponding to an average of 0.47 hallucinated claims per description. We acknowledge this remaining limitation and will incorporate both the new evaluation and a discussion of these findings into the revised manuscript.
[CONTINUED BELOW]
Author Response to Reviewer wjCj (3/3)
[CONTINUED]
W2.3 & Q2 (non-interacting inputs and uncertainty)
We would like to clarify that, as a deliberate simplification of data preparation and model training, PPI2Text is trained to assume that the input protein pair interacts, rather than to predict whether an interaction exists. This is an acknowledged limitation as we already discussed in Section 6. Our engineering choice is mainly motivated by the fact that binary PPI prediction (whether two proteins will interact) is a well-studied task, and recent state-of-the-art models reported 95%-99% accuracy [1-3]. Therefore, in practical real-world settings, PPI2Text can be integrated into a two-stage pipeline, where a binary PPI prediction model first screens input protein pairs for potential interactions, and PPI2Text then generates detailed descriptions for pairs predicted to interact. This allows PPI2Text to focus on its main objective of characterizing interactions and to leverage existing specialized models for interaction pre-screening.
However, this does not mean that PPI2Text is unable to express uncertainty when encountering unfamiliar or weakly supported protein pairs. Our data synthesis pipeline explicitly enforced four evidence-tiered labels (length, verb, mechanism and silence) to control the level of confidence strength and expressiveness of the synthesized descriptions. Since PPI2Text is trained on such data with varying levels of evidence support and confidence, it can learn to adapt its language and express uncertainty when encountering weakly supported cases. Following the reviewer's question, we sampled 300 non-interacting pairs from Negatome 2.0 [4] plus 300 random pairs of C3-hard proteins with no interaction recorded in IntAct or STRING. We fed these pairs to PPI2Text and analyzed the generated descriptions:
| Non-interacting pairs | Interacting pairs | |
|---|---|---|
| Asserts a direct physical interaction | 7.8% | 37.7% |
| Opens with hedged wording | 95.7% | 60.7% |
| Raises location mismatch | 40.8% | 6.3% |
We observed a significant shift toward more cautious and less assertive claims for those non-interacting inputs. Notably, PPI2Text raised potential subcellular location mismatches much more often, suggesting that the model can recognize concrete biological constraints and adapt its wording accordingly. Rather than uniformly asserting interactions with full confidence, PPI2Text can adapt the strength and specificity of its descriptions, and highlight potential biological limitations when definitive conclusions cannot be reached. This is particularly relevant to real-world applications, where many candidate PPIs are poorly characterized and their interaction mechanisms remain unknown. These observations suggest that, with its confidence-aware training, PPI2Text retains its ability to communicate uncertainty, rather than presenting unsupported details as established facts.
Again, we sincerely appreciate the reviewer's constructive feedback, which has helped us improve the clarity and completeness of our work. We hope that our detailed responses and additional evidence adequately address your concerns and further demonstrate the effectiveness and potential of PPI2Text. We will reflect these improvements in the revised paper.
[1] Dang, Thanh Hai, and Tien Anh Vu. "xCAPT5: protein–protein interaction prediction using deep and wide multi-kernel pooling convolutional neural networks with protein language model." BMC bioinformatics 25.1 (2024): 106.
[2] Hu, An, Linai Kuang, and Dinghai Yang. "LPBERT: a protein–protein interaction prediction method based on a pre-trained language model." Applied Sciences 15.6 (2025): 3283.
[3] Zhou, Senyu, et al. "Predicting protein–protein interactions in microbes associated with cardiovascular diseases using deep denoising autoencoders and evolutionary information." Frontiers in Pharmacology 16 (2025): 1565860.
[4] Blohm, Philipp, et al. "Negatome 2.0: a database of non-interacting proteins derived by literature mining, manual annotation and protein structure analysis." Nucleic acids research 42.D1 (2014): D396-D400.
Author Response to Reviewer wjCj (1/3)
Dear Reviewer:
We sincerely thank you for the positive and constructive assessment of our work. We are very much encouraged by your recognition of the well-motivated architecture, the value of our large-scale dataset, and the comprehensive experimental evaluation supporting our proposed components. We greatly appreciate your insightful suggestions, which provide valuable directions for further improving our work. Below, we address each of the weaknesses (W) and questions (Q) raised in your review in detail.
W1 & Q1 (pair map effectiveness)
We would like to clarify that the pair map is a learned feature grid, and is not guaranteed to be a strict "contact map". It is trained only through the captioning loss, and has no supervision on interface positions. Nevertheless, to measure how much interface information the pair map actually carries, we trained a linear classifier on top of each kind of pair map representation to predict the interface location between two proteins:
The result suggests that the learned pair map indeed encodes coarse-grained interface information even though it was never supervised to do so, and that pooling to 32x32 preserves most of the information. We will make this clearer in the paper. Adding an explicit interface supervision signal to the pair map could be a promising direction for future work.
We also conducted the ablations as the reviewer suggested to further demonstrate the contribution of the pair map architecture:
HA,HB are the outputs of the bidirectional cross-attention module, as shown in our Figure 2
Using the output of cross-attention instead of the compressed output of ESM3 does improve the metrics, but the pair map still provides substantially better performance with our PaCo-RoPE design.
We also ablated our PaCo-RoPE design:
The results demonstrate that both coordinate alignment and interleaved frequency allocation are important design choices in our PaCo-RoPE.
W2.1 (hallucination risk in synthesis)
We agree with the reviewer that, despite the evidence-tiered filtering and constraints applied in our data synthesis pipeline, the risk of hallucinations cannot be entirely ignored. To assess such risk, we conducted a new round of human auditing on the synthesized text, where every claim or statement in the synthesized descriptions was individually cross-checked against the sources. 60 PPIs were sampled and examined, with about 14 claims in each description on average. We found that 82% of the statements were directly supported by at least one source, whereas 17% of the claims were biologically plausible generic inferences made by Gemini based on the available evidence. Most importantly, we found only 2 claims (0.2%) that contradict either the source evidence or established biological principles. This further suggests that, with well-designed instruction prompting and comprehensive evidence sources, the synthesis pipeline minimizes factual hallucinations and greatly reduces the risk of introducing misleading information into our model training.
[CONTINUED BELOW]