- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies how to use protein folding models' representations to improve learning dynamics of design model. Specifically, the paper adapts the idea of REPA from image domain to align the pair representations of RFD3 with a MSA-conditioned Bolt-2 teacher. The authors explore two variants of REPA linear project layers: bias-free linear and affine. The authors re-trained the RFD3 as a baseline to compare with the proposed ProteinREPA variants. The results show marginal improvements on three benchmark tasks: enzyme-motif scaffolding, protein-protein and protein-ligand binder design tasks.
Critical strengths
None
Critical weaknesses
- The main weakness of the paper is the experimental results. There is no external baselines for comparison, and the proposed variants are much worse than the published RFD3 on protein-protein and protein-ligand binder design tasks, which makes the method questionable to be used in real world. There are many missing or inconsistent experimental details in the paper, making it hard to reproduce the results.
- Methodological novelty is the other big concern. The paper simply adapts the idea of REPA on protein design models without any methodological innovations or new insights.
Questions for authors
There are several inconsistencies or missing details in the manuscript:
- Protein-binder evaluation uses different predictors. §4.3.2 and Appendix F.3 specify AF3. Table 10’s “This work” row instead specifies AF2 with initial guess.
- The hotspot success requirement contradicts itself. §4.3.2 requires ≥75% of specified hotspots within 5 Å of binder heavy atoms. Appendix F.3 explicitly says hotspot contacts do not affect passing.
- Ligand success thresholds differ. §4.3.3 uses scaffold scRMSD <2 Å, ligand RMSD <5 Å, and minimum inter-chain pAE <2 Å. Appendix F.4 instead uses scRMSD ≤1.5 Å, pAE ≤1.5 Å, and additionally ipTM ≥0.8.
- The distillation-data confidence claim is incorrect as written. §8 says only distillation examples with normalized pLDDT >0.90 are used. Appendix D.2 says no pLDDT gate was applied to AFDB complexes, which have nonzero training probability.
- The steering algorithm is unfinished. Appendix B.4 literally leaves “[CONFIRM: the denoised prediction, the score, or the sampler drift]” where the gradient update should be specified.
Besides, there are significant concerns in the experimental results:
- The default protein-binder benefit is extremely small in aggregate. From Fig. 4, summing all ten targets gives 117 total successes for matched control and 119 for ProteinREPA–affine. The benefits of the method are very target-dependent, and hard to prove a broad protein-binder improvement.
- The ligand improvement is also modest when aggregated. Fig. 5 gives average success rates of 24.0% for the control and 24.6% for affine.
- The different variants including the control are significantly worse than the published RFD3 results, which makes the proposed model hard to use in real world.
- There are no external baselines, e.g., BindCraft, Proteina-Complexa, PXDesign or BoltzGen, which are better models than RFD3.
- Running 10 self-conditioning might slow down the model inference and thus affect the inference-time scaling in the end.
2 comments
Thank you for the clarifications and additional experiments. However, my main concerns about competitiveness against strong external baselines and the method’s practical usefulness remain unresolved. I have increased my score from 1 to 2 to acknowledge the additional evidence and more carefully scoped claims.
We thank the reviewer for the thorough review and apologise for the inconsistencies. These were reporting errors: each benchmark used one configuration, and the text now matches it. All changes are highlighted in blue.
Q1. Inconsistencies.
We corrected all five:
Binder predictor. All binder evaluations used AlphaFold 3 (AF3), AlphaProteo targets and AF3 success thresholds. We corrected the "This work" row in App. F.3, Table 11 (previously Table 10).
Hotspots. Hotspot residues were used for conditioning and recorded as a diagnostic, but excluded from success criteria. We clarified §4.3.2 to match App. F.3.
Ligand thresholds. Following RFD3: scRMSD ≤ 1.5 Å, ligand RMSD ≤ 5 Å, minimum inter-chain pAE ≤ 1.5 Å, ipTM ≥ 0.8. We corrected §4.3.3 to match App. F.4, which was already correct.
pLDDT filtering. AFDB monomers required mean pLDDT ≥ 90; complexes had no pLDDT filter. We corrected §8 to match App. D.2.
Steering update. The gradient modifies predicted clean coordinates; the guided prediction enters the subsequent sampler update. We clarified the procedure and added the complete update rule in App. B.4.
Q2. Aggregate binder and ligand effects.
Easy targets dominate pooled counts: BHRF1 contributes 70 of the control's 117 passing binder backbones. We added per-target Fisher's exact tests, Holm–Bonferroni correction across targets within each benchmark, and sign tests across targets. The following per-target results compare each model with the matched control; p-values are from sign tests.
Enzyme (41 targets). Affine improves on 35 and worsens on 4 (p = 3 × 10⁻⁷), with 15 improvements significant after correction. Linear improves on 20 and worsens on 10 (p = 0.10).
Binder (10 targets). Affine improves on 2 and worsens on 4 (p = 0.69); linear improves on 6 and worsens on 1 (p = 0.13). For both models, only InsulinR remains significant after correction.
Ligand (4 targets). Affine improves on 3 and worsens on 1, with no individually significant differences.
These results support a broad enzyme gain. Binder and ligand effects are target-dependent and not established for the affine model; we narrowed those claims accordingly.
Where: App. F.5 (new method, summary table and per-target tables). Claims narrowed in the abstract, §1, §4.3.2–4.3.3, the Synthesis paragraph of §4.3 and §5.
Q3. Published RFD3 and external baselines.
Our primary question is whether alignment improves generation under controlled training conditions, which the matched control addresses. External baselines address competitiveness. Published RFD3, trained on data unavailable to us, remains stronger on several benchmarks. It is shown for reference only; we do not claim to outperform it.
We added CATH/SHAPES and MotifBench evaluations, benchmarks used by ReaPro-1c, with Protpardelle-1c as a shared reference for CATH/SHAPES. This compares alignment effects relative to each method's own control, while acknowledging differences in architectures and training regimes.
Where: §4.1, App. C (limitations of external comparisons), App. F.8–F.9 (new evaluations).
Q4. Inference cost of increased self-conditioning.
We now quantify the performance–cost trade-off from additional iterations. For the affine model, increasing from two to ten iterations raises the mean enzyme pass rate across targets from 25.0% to 35.6%, improving on 33 of 41 targets. This suggests further refinement can exploit the learned representations.
Backbone sampling time rises from 1.0 to 3.9 s/design at 100 residues and from 3.8 to 14.8 s/design at 300 residues, approximately fourfold (one RTX PRO 6000 GPU; excluding sequence design and AF3 evaluation). Standard generation retains two iterations, with no additional inference cost relative to RFD3. The submission tested ten iterations only for the affine model; evaluations for the control, linear and published RFD3 models are ongoing. We will report the results in the discussion once available.
Where: App. F.6 (new, ten iterations for all models), App. F.7 (new, sampling cost) and App. C (limitations).
Q5. Novelty.
ProteinREPA differs in both the aligned representation and supervision source. ReaPro-1c aligns residue-level features with pretrained protein language or inverse-folding models; DMRA uses amino-acid-type embeddings for residue-level alignment. ProteinREPA aligns pairwise representations with those of a complete, MSA-conditioned structure predictor, transferring structural and molecular context into an all-atom generator without additional inference cost. Under matched training, this improves sequence–structure compatibility and enzyme motif scaffolding.
Where: §1 (novelty and related work), §3 (alignment methodology) and App. F.8 (new ReaPro-1c comparison). See also our response to reviewer npsG.