acceptodds
Under review as a conference paper at ICLR 2027

ProteinREPA: Structure Predictors as Representation Teachers for Generative Protein Design

Abstract

Modern protein design commonly follows a design-and-filter strategy, in which generative models propose candidates and biomolecular structure predictors determine which designs advance. In this paradigm, the predictor’s knowledge is primarily used to reject unsuccessful designs rather than to improve the generative process itself. We ask whether internal representations encoding geometric and evolutionary priors can be instead learned by generative protein design models and, if so, which capabilities improve. We introduce ProteinREPA, which aligns the pair representation of RFD3 with that of a frozen, MSA-conditioned Boltz-2 teacher using a lightweight learned projection and cosine-similarity objective. The teacher is used only during training and introduces no additional cost during standard inference. We compare ProteinREPA with matched unaligned controls trained using the same data and optimization schedule, evaluating unconditional sequence–structure generation, protein–protein binder design, protein–ligand binder design, and enzyme motif scaffolding. ProteinREPA improves intrinsic sequence–structure compatibility: the fraction of jointly generated sequence–structure pairs that refold self-consistently rises from 34.5% for the matched control to 43.8% with the affine projection and 51.3% with the bias-free linear projection. Furthermore, training RFD3 with ProteinREPA improves the success rates for the majority of task specific design benchmark cases, such as enzyme–motif scaffolding, protein–protein-, and protein–ligand-binding tasks, demonstrating that alignment also improves functional scaffold generation. Increasing coordinate-mediated self-conditioning from two to ten iterations provides a further and often larger gain across the three benchmark classes; for example, the number of passing BHRF1 binder backbones increases from 63 to 125 out of 200. Together, these results establish biomolecular structure predictors as representation teachers for generative protein models and show that improved learned representations and inference-time geometric search address complementary limitations of current all-atom generators.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein design, protein structure prediction, generative models, diffusion models, representation alignment, knowledge distillation, sequence–structure co-design, computational biology

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1N61OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how to use protein folding models' representations to improve learning dynamics of design model. Specifically, the paper adapts the idea of REPA from image domain to align the pair representations of RFD3 with a MSA-conditioned Bolt-2 teacher. The authors explore two variants of REPA linear project layers: bias-free linear and affine. The authors re-trained the RFD3 as a baseline to compare with the proposed ProteinREPA variants. The results show marginal improvements on three benchmark tasks: enzyme-motif scaffolding, protein-protein and protein-ligand binder design tasks.

Critical strengths

None

Critical weaknesses
  1. The main weakness of the paper is the experimental results. There is no external baselines for comparison, and the proposed variants are much worse than the published RFD3 on protein-protein and protein-ligand binder design tasks, which makes the method questionable to be used in real world. There are many missing or inconsistent experimental details in the paper, making it hard to reproduce the results.
  2. Methodological novelty is the other big concern. The paper simply adapts the idea of REPA on protein design models without any methodological innovations or new insights.
Questions for authors

There are several inconsistencies or missing details in the manuscript:

  1. Protein-binder evaluation uses different predictors. §4.3.2 and Appendix F.3 specify AF3. Table 10’s “This work” row instead specifies AF2 with initial guess.
  2. The hotspot success requirement contradicts itself. §4.3.2 requires ≥75% of specified hotspots within 5 Å of binder heavy atoms. Appendix F.3 explicitly says hotspot contacts do not affect passing.
  3. Ligand success thresholds differ. §4.3.3 uses scaffold scRMSD <2 Å, ligand RMSD <5 Å, and minimum inter-chain pAE <2 Å. Appendix F.4 instead uses scRMSD ≤1.5 Å, pAE ≤1.5 Å, and additionally ipTM ≥0.8.
  4. The distillation-data confidence claim is incorrect as written. §8 says only distillation examples with normalized pLDDT >0.90 are used. Appendix D.2 says no pLDDT gate was applied to AFDB complexes, which have nonzero training probability.
  5. The steering algorithm is unfinished. Appendix B.4 literally leaves “[CONFIRM: the denoised prediction, the score, or the sampler drift]” where the gradient update should be specified.

Besides, there are significant concerns in the experimental results:

  1. The default protein-binder benefit is extremely small in aggregate. From Fig. 4, summing all ten targets gives 117 total successes for matched control and 119 for ProteinREPA–affine. The benefits of the method are very target-dependent, and hard to prove a broad protein-binder improvement.
  2. The ligand improvement is also modest when aggregated. Fig. 5 gives average success rates of 24.0% for the control and 24.6% for affine.
  3. The different variants including the control are significantly worse than the published RFD3 results, which makes the proposed model hard to use in real world.
  4. There are no external baselines, e.g., BindCraft, Proteina-Complexa, PXDesign or BoltzGen, which are better models than RFD3.
  5. Running 10 self-conditioning might slow down the model inference and thus affect the inference-time scaling in the end.
2 comments

We thank the reviewer for the thorough review and apologise for the inconsistencies. These were reporting errors: each benchmark used one configuration, and the text now matches it. All changes are highlighted in blue.

Q1. Inconsistencies.

We corrected all five:

Binder predictor. All binder evaluations used AlphaFold 3 (AF3), AlphaProteo targets and AF3 success thresholds. We corrected the "This work" row in App. F.3, Table 11 (previously Table 10).

Hotspots. Hotspot residues were used for conditioning and recorded as a diagnostic, but excluded from success criteria. We clarified §4.3.2 to match App. F.3.

Ligand thresholds. Following RFD3: scRMSD ≤ 1.5 Å, ligand RMSD ≤ 5 Å, minimum inter-chain pAE ≤ 1.5 Å, ipTM ≥ 0.8. We corrected §4.3.3 to match App. F.4, which was already correct.

pLDDT filtering. AFDB monomers required mean pLDDT ≥ 90; complexes had no pLDDT filter. We corrected §8 to match App. D.2.

Steering update. The gradient modifies predicted clean coordinates; the guided prediction enters the subsequent sampler update. We clarified the procedure and added the complete update rule in App. B.4.

Q2. Aggregate binder and ligand effects.

Easy targets dominate pooled counts: BHRF1 contributes 70 of the control's 117 passing binder backbones. We added per-target Fisher's exact tests, Holm–Bonferroni correction across targets within each benchmark, and sign tests across targets. The following per-target results compare each model with the matched control; p-values are from sign tests.

Enzyme (41 targets). Affine improves on 35 and worsens on 4 (p = 3 × 10⁻⁷), with 15 improvements significant after correction. Linear improves on 20 and worsens on 10 (p = 0.10).

Binder (10 targets). Affine improves on 2 and worsens on 4 (p = 0.69); linear improves on 6 and worsens on 1 (p = 0.13). For both models, only InsulinR remains significant after correction.

Ligand (4 targets). Affine improves on 3 and worsens on 1, with no individually significant differences.

These results support a broad enzyme gain. Binder and ligand effects are target-dependent and not established for the affine model; we narrowed those claims accordingly.

Where: App. F.5 (new method, summary table and per-target tables). Claims narrowed in the abstract, §1, §4.3.2–4.3.3, the Synthesis paragraph of §4.3 and §5.

Q3. Published RFD3 and external baselines.

Our primary question is whether alignment improves generation under controlled training conditions, which the matched control addresses. External baselines address competitiveness. Published RFD3, trained on data unavailable to us, remains stronger on several benchmarks. It is shown for reference only; we do not claim to outperform it.

We added CATH/SHAPES and MotifBench evaluations, benchmarks used by ReaPro-1c, with Protpardelle-1c as a shared reference for CATH/SHAPES. This compares alignment effects relative to each method's own control, while acknowledging differences in architectures and training regimes.

Where: §4.1, App. C (limitations of external comparisons), App. F.8–F.9 (new evaluations).

Q4. Inference cost of increased self-conditioning.

We now quantify the performance–cost trade-off from additional iterations. For the affine model, increasing from two to ten iterations raises the mean enzyme pass rate across targets from 25.0% to 35.6%, improving on 33 of 41 targets. This suggests further refinement can exploit the learned representations.

Backbone sampling time rises from 1.0 to 3.9 s/design at 100 residues and from 3.8 to 14.8 s/design at 300 residues, approximately fourfold (one RTX PRO 6000 GPU; excluding sequence design and AF3 evaluation). Standard generation retains two iterations, with no additional inference cost relative to RFD3. The submission tested ten iterations only for the affine model; evaluations for the control, linear and published RFD3 models are ongoing. We will report the results in the discussion once available.

Where: App. F.6 (new, ten iterations for all models), App. F.7 (new, sampling cost) and App. C (limitations).

Q5. Novelty.

ProteinREPA differs in both the aligned representation and supervision source. ReaPro-1c aligns residue-level features with pretrained protein language or inverse-folding models; DMRA uses amino-acid-type embeddings for residue-level alignment. ProteinREPA aligns pairwise representations with those of a complete, MSA-conditioned structure predictor, transferring structural and molecular context into an all-atom generator without additional inference cost. Under matched training, this improves sequence–structure compatibility and enzyme motif scaffolding.

Where: §1 (novelty and related work), §3 (alignment methodology) and App. F.8 (new ReaPro-1c comparison). See also our response to reviewer npsG.

Thank you for the clarifications and additional experiments. However, my main concerns about competitiveness against strong external baselines and the method’s practical usefulness remain unresolved. I have increased my score from 1 to 2 to acknowledge the additional evidence and more carefully scoped claims.

Reviewer_npsGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ProteinREPA, a representation alignment framework that transfers structural and evolutionary information from a frozen, MSA-conditioned Boltz-2 teacher to the RFD3 generative model. By aligning intermediate pair representations through a learned projection and cosine-similarity loss, ProteinREPA improves sequence–structure compatibility without introducing additional inference costs. Experiments show gains in unconditional generation, enzyme motif scaffolding, and protein–ligand binder design. However, improvements in protein–protein binder design are limited and target-dependent, while additional self-conditioning yields further gains. Overall, the work demonstrates the potential of representation alignment for protein design, although its methodological novelty relative to existing REPA-based approaches warrants further clarification.

Critical strengths
  • The paper introduces a simple yet principled framework that aligns RFD3's pair representations with those of a frozen, MSA-conditioned Boltz-2 teacher. This allows structural and evolutionary information to improve the generator without introducing additional inference costs.
  • The authors evaluate ProteinREPA on unconditional sequence–structure generation, enzyme motif scaffolding, protein–protein binder design, and protein–ligand binder design. The results demonstrate consistent improvements in enzyme motif scaffolding and gains on most protein–ligand targets, suggesting that representation alignment can benefit more than sequence–structure compatibility alone.
Critical weaknesses

Limited comparison with existing representation alignment methods. The core idea of aligning intermediate representations of protein generative models with pretrained protein models has been explored in prior work, including ReaPro-1c and DMRA. However, ProteinREPA does not compare against these approaches, making it difficult to assess its methodological novelty and the specific benefits of using MSA-conditioned pair representations from a structure predictor. Comparisons with these methods would help establish the added value of ProteinREPA over existing representation alignment strategies.

Questions for authors

Could the authors provide a comparative analysis against at least one existing method based on the same REPA paradigm to better demonstrate the algorithmic advantages of ProteinREPA?

1 comment

We thank the reviewer for the positive assessment and for pointing us to ReaPro-1c and DMRA. All changes are highlighted in blue in the revised PDF.

Q1. Novelty relative to REPA-based methods.

We show that a complete structure predictor's pair representation is a useful training signal for an all-atom generator. Under matched training, it improves the compatibility of jointly generated sequences and structures (designability 34.5% → 43.8%) and enzyme motif scaffolding (affine better on 35 of 41 targets), without adding inference cost. The two cited works instead align residue-wise features to a pretrained representation. ReaPro-1c performs backbone generation and motif scaffolding, aligning Protpardelle-1c's per-residue hidden representations with those of pretrained ProteinMPNN or ESM2.

DMRA performs diffusion-based protein inverse folding, aligning residue-level hidden representations with amino-acid-type embeddings during denoising.

ProteinREPA introduces a new kind of teacher for representation alignment in protein generation. It aligns pair rather than residue features, and it draws them from a complete MSA-conditioned structure predictor rather than a protein language or inverse-folding model.

Where: §1 cites both works. App. C (Limitations) explains why a head-to-head comparison is limited, and App. F.8 (new) compares with ReaPro-1c on its own benchmark.

Q2. Comparison with these methods. We ran our models on ReaPro-1c's own two benchmarks.

A head-to-head comparison of the methods is not possible without new training runs, because base models, training data and tasks differ, and DMRA does inverse folding, which RFD3 does not.

CATH/SHAPES (new, App. F.8). Relative to the matched control, ProteinREPA-affine raises FPD from 0.56 to 0.63 (layer 1) and from 0.32 to 0.34 (layer 3), while ReaPro-1c lowers it (0.36 → 0.29 and 0.23 → 0.21). Because base models, data and training pipelines differ, we cannot attribute the shift in FPD to the teachers.

MotifBench (new, App. F.9). MotifBench scaffolds backbone motifs without side chains or ligands. Here we detect no improvement from alignment. The affine model and the control each solve 24 of 30 problems and score 33.6 and 34.7, and the per-problem success rate is higher for the affine model on 12 problems, lower on 11 and tied on 7. This contrasts with our all-atom enzyme benchmark, where catalytic side chains must be placed around a bound ligand and the affine model is better on 35 of 41 targets. The benefit of alignment thus appears where all-atom and ligand context matter, which fits a teacher that represents this context. We regard this as a hypothesis, not a tested mechanism.

Where: App. F.8 (new, CATH/SHAPES comparison, including FPD, precision, recall, and coverage in Table 19, and UMAP visualization in Fig. 13) and App. F.9 (new, MotifBench results in Table 20).

Reviewer_1hVEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper applies the representation alignment framework to align RFD3's pair representation with that of Boltz-2 with cosine similarity loss. Empirically, it improves unconditional codesignability and enzyme motif scaffolding success rates.

Critical strengths

The paper demonstrated the effectiveness of representation alignment in protein generative modeling. ProteinREPA is a simple and effective approach to transferring knowledge from structure predictors to protein generative models without additional inference cost.

Critical weaknesses
  1. The paper could benefit from additional experiments to better understand what knowledge is actually transferred through representation alignment. For example, it is currently unclear whether the improvements primarily come from Boltz-2's learned structural priors or from the additional evolutionary information provided by MSA. Although the authors analyze how MSA depth affects the teacher representations, they do not show whether this information actually improves the downstream generation performance. Could the authors compare Boltz-2 teachers with and without MSAs to determine the contribution of evolutionary information? Also, additional comparisons with alternative pretrained protein representations would help show whether ProteinREPA generalizes beyond the particular teacher used in the paper.

  2. The manuscript is incomplete. The paper contains numerous question marks in place of missing citations, as well as leftover TODO. I encourage the authors to take greater responsibility for proofreading the manuscript before submission.

Questions for authors

See Critical Weaknesses.

1 comment

We thank the reviewer for the constructive review and helpful suggestions. All changes are highlighted in blue in the revised PDF.

Q1. Structural priors vs. MSA information. The teacher's pair representation carries mostly structural information. When Boltz-2 is given the native structure as a template without an MSA, or the MSA without a template, the two pair representations are nearly identical within chains (CKA 0.98 on 16 protein–ligand systems, 0.94 on 31 complexes). They differ mainly in the inter-chain region (CKA 0.74 at interfaces). Interface pairs are a small share of the aligned pairs, so the alignment signal is mainly structural. Training an additional checkpoint using MSA-free teacher representations was unfortunately beyond our budget. It requires new teacher representations for 469,317 training examples (about 10k GPU-hours), retraining (about 4.5k GPU-hours) and benchmarking (about 1k GPU-hours).

Where: App. B.5 (new), App. B.3, and App. C (Limitations), which now states the cost of an MSA-free teacher.

Q2. Additional comparisons with alternative pretrained protein representations. Retraining RFD3 with the teacher representations of the cited works was beyond our compute budget, because each experiment requires its own representation cache and training. The closest available comparison is to measure the relative performance difference of our aligned models and the control on a benchmark shared with another REPA method. We evaluated our models on the CATH/SHAPES benchmark of ReaPro-1c, which aligns Protpardelle-1c to ProteinMPNN and ESM2. Relative to the matched control, our alignment substantially improves designability (34.5% → 43.8%) while slightly increasing the distance to CATH (FPD 0.56 → 0.63, layer 1, lower is closer), whereas ReaPro-1c's alignment reduces this distance (0.36 → 0.29). This difference reflects the distinct priorities of the two approaches: while matching the natural protein distribution is an important objective in ReaPro-1c, our primary goal is to improve designability rather than minimize FPD. We therefore view the substantial gain in designability as a favorable trade-off for the modest increase in distributional distance.

MotifBench (App. F.9, Table 20). Unlike our all-atom enzyme benchmark, MotifBench evaluates backbone motif scaffolding without side-chain or ligand constraints. On this benchmark, we find no clear benefit from alignment: the affine model and the control both solve 24 of 30 problems, with scores of 33.6 and 34.7, respectively. Across individual problems, the affine model achieves higher success rates on 12, lower rates on 11, and ties on 7. In contrast, on our all-atom enzyme benchmark, which requires positioning catalytic side chains around a bound ligand, the affine model outperforms the control on 35 of 41 targets. These results suggest that alignment may be particularly beneficial for tasks requiring all-atom and ligand context, consistent with the information captured by our Boltz-2 teacher, although this remains a hypothesis rather than an established mechanism.

Together, these results suggest that the benefits of alignment depend on the task and evaluation criteria. In particular, our alignment improves designability despite slightly increasing the distance to the natural protein distribution, whereas ReaPro-1c reduces this distance. Because the base models, training data, and evaluation pipelines differ, we cannot attribute these contrasting effects to the choice of teacher alone.

Where: App. F.8 (new) presents the CATH/SHAPES comparison with ReaPro-1c, and App. F.9 (new, Table 20) reports the MotifBench results. Designability improvements are documented in §4.2 (Fig. 2b) and App. Table 9. Evaluation limitations and comparisons with alternative teachers are discussed as future work in App. C.

Q3. Missing citations and leftover placeholder. We apologise. All unresolved citations are fixed, the placeholder in App. B.4 is replaced by the full update rule, and we checked the manuscript against our evaluation configurations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.