- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents NLPro, a protein generation framework conditioned on natural-language functional descriptions and biological context such as ligands and antigens. It first constructs NLProBench and pretrain a protein decoder using autoregressive generation and span-infilling objectives. The pretrained model is subsequently adapted to antibody and ligand-binding protein design through gated cross-attention to specialized modality encoders. Computational evaluations report improvements in structural accuracy, InterPro recovery, antibody DockQ, and ligand pocket geometry over several existing baselines.
Critical strengths
1. The paper presents a large-scale and practically relevant protein design framework. NLPro unifies natural-language functional descriptions, ligand information, and antigen context within a single generative architecture, and evaluates the framework across various tasks. This broad scope makes the work relevant to both protein language modeling and conditional biological design.
2. The dataset construction and experimental documentation are relatively comprehensive. The authors construct a large protein-text corpus, apply cluster-level train-test splitting, compare against several categories of baselines, and provide detailed descriptions of the architecture, training configuration, evaluation metrics, and baseline settings. These resources and implementation details could support subsequent research on instruction-guided protein generation.
Critical weaknesses
- The benefit of the biological-context pathway appears limited and task-dependent. Figure 3 shows that ligand conditioning improves pocket RMSD and docking success, but has little effect on pLDDT, ipTM, or predicted binding affinity. More notably, antigen conditioning does not improve antibody design, with the text-only variant achieving a slightly higher DockQ than the text-plus-modality variant. Since integrating biological context is a central component of NLPro, the inconsistent gains make its contribution less convincing.
- The statistical reliability of several reported improvements is insufficiently established. Some performance differences are small, such as 0.39 versus 0.38 DockQ in antibody design and 81.8% versus 80.8% success on unseen ligands, while the paper does not report variation across training runs or confidence intervals. Without uncertainty estimates or significance tests, it is difficult to determine whether these improvements are stable.
Questions for authors
- Could the authors further analyze why antigen conditioning does not improve antibody design? In addition, could they compare correct and mismatched ligand or antigen inputs to determine whether NLPro is sensitive to the specific biological context provided?
- Could the authors repeat the main downstream experiments across multiple training seeds and report confidence intervals, together with paired bootstrap significance tests over evaluation targets, particularly for the antibody and unseen-ligand results?
- Could the authors report the sequence and structural similarity between generated proteins and their nearest neighbors in the training set, together with diversity across repeated generations for the same instruction? This would help clarify whether the model generates novel candidates rather than close variants of known proteins.
2 comments
Dear Reviewer,
We have added experiments and addressed all your concerns. As the rebuttal deadline is approaching, would you please take a look at our response? If you have any additional questions, we are more than happy to discuss.
Best,
Authors of paper 21634
Q1. Why antigen conditioning does not improve antibody design, and whether NLPro is sensitive to the specific context provided
Figure 3's text-only DockQ was computed on an incomplete target subset. Rescored on a common 433 targets the two variants are indistinguishable, 0.383 against 0.381, and the heavy-antigen interface is unchanged (−0.003, 95% CI [−0.017, +0.012], p=0.75).
Antigen conditioning changes whether AlphaFold3 places an antibody-antigen interface, not the quality of the interfaces it places. DockQ is defined only where an interface exists, so we use ipSAE (Dunbrack, 2025), which scores a chain pair from the predicted aligned error on every prediction.
Does AlphaFold3 place an antigen interface?
A mismatched antigen recovers about half the gain, so part of what the pathway supplies is a generic prior that a partner is present. The correct antigen beats the mismatched one on interface confidence (+0.012, p=0.028), so the rest is partner-specific. On the same 433 targets ipTM rises from 0.517 to 0.530 (p=0.008) and pLDDT from 85.68 to 86.11 (p=0.002).
Q2. Confidence intervals and paired bootstrap significance tests
Paired bootstrap over targets, 10,000 resamples, both models moving together, with exact McNemar tests for binary outcomes.
The averaged antibody DockQ does not separate from zero, which is the 0.39 against 0.38 cited. Decomposing by interface shows why: we lead on the antigen interface and trail on heavy-light, which the supplied framework fixes rather than the designed loops.
Antibody, NLPro minus RefineGNN
Docking success, pocket RMSD below 2 Å. NLPro minus baseline, percentage points
The 81.8% against 80.8% cited is the ProGen2 unseen comparison. Retraining from several seeds does not fit the discussion period, so the uncertainty is over targets rather than training runs.
Q3. Similarity to training-set nearest neighbours, and diversity across repeated generations
All designs use nucleus sampling, p=0.95 and τ=0.8.
Ligand designs against the 205.7M-sequence pretraining corpus
Against the stage-2 fine-tuning corpus, 39,488 sequences, no design reproduces a sequence exactly, and our mean identity of 54.9% against the ground truth's 52.4% is matched by Pinal at 54.2%, which never saw this corpus.
Ligand designs against their closest match in AFDB/UniProt50, 54M structures
Antibody. The framework is supplied and only the CDR loops are generated, so we measured the loops. CDR-H3 recovery against the target loop is 36.8% for NLPro and 35.8% for RefineGNN, with 2 of 438 designs reproducing the target exactly. Against the fine-tuning corpus, 13.6% of our loops of length 12 or more occur verbatim in training, against 0 of 349 for both the ground truth loops and RefineGNN. The ground truth rate of zero shows the split held, so this is memorisation rather than leakage, and we report it as a limitation. It appears specific to loop infilling, where the output space is short and the corpus redundant. On the ligand task no design in any arm reproduces a fine-tuning sequence verbatim.
Diversity. Three decoding seeds per instruction. No ligand design repeats between seeds, 0 of 126 unseen. On antibodies 87.4% of targets yield at least two distinct CDR-H3 loops and 63.5% yield three.