- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SitePepGen jointly learns peptide sequence generation and receptor binding-site prediction using a shared ESM-2 backbone. Candidates are ranked by sequence confidence and compatibility with predicted sites, without requiring receptor structures or predefined binding regions. For each generated peptide, LocalColabFold predicts a corresponding receptor-peptide complex. Results show improved receptor binding-site recovery over PepMLM.
Critical strengths
-
Incorporating receptor binding-site information addresses a relevant limitation of sequence-conditioned peptide design. Generation and candidate selection require neither receptor structures nor predefined binding regions, making the approach applicable when such information is unavailable.
-
Joint training connects residue-level site prediction with peptide generation through a shared ESM-2 backbone, with little additional parameter overhead. Site-aware candidate ranking also avoids complex-structure prediction within the selection loop.
-
Experimental results show consistent improvements in receptor binding-site recovery over the evaluated baselines. Compared with PepMLM, SitePepGen improves macro Jaccard and F1 by 3.62 and 3.56 percentage points, respectively, with paired bootstrap confidence intervals supporting both gains. Separate site-prediction results and sequence-plausibility diagnostics provide complementary evidence for the approach.
Critical weaknesses
-
Among 202 test queries, 74 have receptor sequences identical to those in training, while unseen receptors are defined solely by the absence of exact sequence matches. Evaluation using receptor sequence clustering is needed to assess generalization beyond closely related proteins.
-
The compatibility term in Eq. (4) summarizes amino-acid composition, hydrophobicity, and charge, without representing residue ordering or geometric interactions. Peptides with identical compositions but different residue arrangements receive identical compatibility scores. Its effectiveness as a measure of site-specific binding compatibility requires stronger validation.
-
Ablation experiments lack a comparison with PepMLM using the same imputation procedure, as well as a comparison that ranks the same candidate pool using only language-model scores. These controls would help isolate the contribution of receptor binding-site information.
-
Structural metrics are averaged over each method’s structurally evaluable queries, but the corresponding sample counts and exclusion reasons are not clearly reported. If these subsets differ across methods, direct comparisons may be biased.
Questions for authors
- How similar are the test receptors to those in training beyond exact sequence matches? Do the gains persist under a split based on receptor sequence clustering?
- What supports the physicochemical compatibility score in Eq. (4), particularly the amino-acid composition term? Does using predicted sites outperform using random or shuffled sites?
- Could you include PepMLM with the same imputation procedure and a baseline that ranks the same candidate pool using only language-model scores? These comparisons would help isolate the contribution of receptor binding-site information.
- How many queries are structurally evaluable for each method, and how are unsuccessful predictions handled? Do the gains persist when all methods are evaluated on the same query subset?
7 comments
Compatibility Score Motivation and Ranking Controls
Q2:What supports the physicochemical compatibility score in Eq. (4), particularly the amino-acid composition term? Does using predicted sites outperform using random or shuffled sites?
Response:
Thank you for this suggestion. Eq. (4) uses amino-acid compositional overlap, hydrophobicity, and charge to provide coarse site-level physicochemical information for auxiliary candidate ranking. Composition-based descriptors have been used for peptide–protein interaction prediction [1], and mutational studies have shown the roles of hydrophobicity, electrostatics, and residue identity in peptide recognition [2,3]. These studies motivate the descriptor choices.
We added two inference-time ranking controls. Within each run, all configurations use the same site-supervised SitePepGen checkpoint, the same candidate sequences, and their associated LM scores.
SitePepGen (LM-only+imputation) removes the entire site-compatibility term in Eq. (4) from ranking, selects candidates using only language-model (LM) scores, and subsequently performs X imputation. This control assesses whether adding the complete compatibility score improves candidate selection.
SitePepGen (random-site control) retains the original compatibility function and component weights but replaces predicted sites with an equal number of residues sampled uniformly without replacement from the same receptor. Composition-, hydrophobicity-, and charge-related compatibility scores are then recomputed using these random sites. This control assesses whether predicted sites provide more useful scoring information than random sites. Both site conditions use identical uniform normalization and empty-site handling. Each query receives one random-site draw per run.
SitePepGen uses the paper’s original configuration, combining LM scoring with compatibility computed from predicted sites. All three configurations retain identical X-imputation, structure-prediction, and contact-evaluation protocols. Results are averaged across paired runs on the same 202 queries.
Table 2. Ranking controls on the same SitePepGen candidate pool.
| Method | Jaccard (%) | Precision (%) | Recall (%) | F1 (%) |
|---|---|---|---|---|
| SitePepGen (LM-only+imputation) | 49.84 | 61.56 | 63.71 | 60.70 |
| SitePepGen (random-site control) | 49.44 | 61.30 | 63.80 | 60.68 |
| SitePepGen | 50.28 | 61.83 | 64.58 | 61.26 |
Adding predicted-site compatibility to LM-only ranking increases Jaccard from 49.84% to 50.28% (+0.44 percentage points) and F1 from 60.70% to 61.26% (+0.56 percentage points), supporting the empirical utility of Eq. (4) as a complete auxiliary ranking feature.
With the compatibility function retained, using predicted sites exceeds using random sites by 0.84 percentage points in Jaccard and 0.58 percentage points in F1, supporting the role of predicted-site information in candidate selection. These experiments assess the complete compatibility score, while the cited literature motivates the composition descriptor.
References
[1] Anindya AL et al. (2024). Deciphering peptide-protein interactions via composition-based prediction: a case study with survivin/BIRC5. Machine Learning: Science and Technology, 5, 025081. DOI
[2] Kruse T et al. (2024). Substrate recognition principles for the PP2A-B55 protein phosphatase. Science Advances, 10, eadp5491. DOI
[3] Benz C et al. (2025). Defining short linear motif binding determinants by phage display-based deep mutational scanning. Protein Science, 34, e70174. DOI
Matched PepMLM and LM-Only Ranking Controls
Q3:Could you include PepMLM with the same imputation procedure and a baseline that ranks the same candidate pool using only language-model scores? These comparisons would help isolate the contribution of receptor binding-site information.
Response:
Thank you for this suggestion. We added PepMLM with the same imputation procedure and an LM-only ranking control on the same SitePepGen candidate pool. Results are reported in Table 3.
PepMLM (LM-only+imputation) uses the same PepMLM model as the main table. It independently generates candidates with a candidate budget and sampling settings matched to SitePepGen, applies LM-only ranking, and performs one-pass X imputation.
SitePepGen (LM-only+imputation) uses candidates generated by SitePepGen, ranks them solely by LM scores, and applies the same X-imputation procedure as in the paper. SitePepGen uses the paper’s main configuration, combining LM scoring with predicted-site compatibility for candidate ranking and following the original X-imputation and structural-evaluation procedures.
The two SitePepGen ranking configurations use the same checkpoint and candidate pool within each run, with identical X-imputation, structure-prediction, and contact-evaluation protocols. Only the inclusion of site compatibility in ranking changes, and results are averaged across paired runs. All configurations are evaluated on the same 202 queries.
Table 3. Matched imputation and LM-only ranking controls.
| Method | Jaccard (%) | Precision (%) | Recall (%) | F1 (%) |
|---|---|---|---|---|
| PepMLM (LM-only+imputation) | 47.15 | 58.78 | 61.21 | 58.10 |
| SitePepGen (LM-only+imputation) | 49.84 | 61.56 | 63.71 | 60.70 |
| SitePepGen | 50.28 | 61.83 | 64.58 | 61.26 |
Under the common LM-only ranking and imputation pipeline, SitePepGen exceeds PepMLM by 2.69 percentage points in Jaccard, with F1 increasing from 58.10% to 60.70% (+2.60 percentage points). This reflects the model-level difference after matching inference.
Adding site compatibility on the same SitePepGen candidate pool increases Jaccard from 49.84% to 50.28% (+0.44 percentage points) and F1 from 60.70% to 61.26% (+0.56 percentage points), reflecting the additional effect of predicted-site information in candidate ranking.
Receptor Similarity Stratification and Subset Evaluation
Q1:How similar are the test receptors to those in training beyond exact sequence matches? Do the gains persist under a split based on receptor sequence clustering?
Response:
Thank you for this suggestion. We added receptor-sequence similarity stratification of the original test set using the original training set and trained models. This is a test-subset analysis; a separate receptor-cluster-disjoint split and retraining have not been performed.
We searched the 6,796 unique training receptor sequences using MMseqs2. A high-similarity training neighbor must satisfy local identity ≥40%, coverage of the shorter sequence ≥70%—with both coverages ≥70% for equal-length sequences—and E-value ≤0.001. Among the original 128 unseen queries, 114 had a detected training neighbor meeting these criteria and 14 did not. Grouping used receptor sequences only. Results are reported in Table 1.
Table 1. Results on 14 queries without detected high-similarity training neighbors.
SitePepGen achieves 1.56 percentage points higher F1 on these 14 queries, providing supplementary evidence for site recovery on this subset.