acceptodds
Under review as a conference paper at ICLR 2027

Self-Supervised Learning of Epistatic Mutation Effects in Structure-based Models

Abstract

Predicting the impact of multiple point mutations on a given protein sequence remains a central challenge in protein engineering, in large part due to the nonlinear effects of epistasis. Experimental data sets that measure multiple mutations are scarce, motivating methods that learn from structural and evolutionary data. We introduce EvoEpi, a self-supervised framework for scoring double mutations from protein sequences and backbone structures without using phenotype labels in the training objective. EvoEpi learns from joint amino-acid frequencies in multiple sequence alignments (MSAs), retaining dependencies between residue pairs that single-position supervision discards. More specifically, our structure-based model predicts changes in log-frequency ratios resulting from double mutations. Trained on the single-chain subset of the ProteinMPNN training set and their corresponding MSAs, EvoEpi achieves global/mean per-protein Spearman correlations of , , and on ThermoMutDB-D, Megascale-D, and PTMul-D, respectively, outperforming all evaluated self-supervised baselines and reaching supervised-level performance, with higher correlations than all evaluated supervised predictors on ThermoMutDB-D and PTMul-D.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Epistasis, protein representation learning, protein engineering, MSA, self-supervised learning, protein stability prediction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nCRNOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

Propose a self-supervised learning scheme for protein mutation effect prediction combining ProteinMPNN, pair state formation, and transition ratio target. Evaluated at ddG datasets and some DMS assays.

Critical strengths
  1. EvoEpi (additive) -> EvoEpi performance improvement (Table 1)
  2. ProteinMPNN redesign of sequence to augment the dataset beyond WT. Ablated in Appendix C.2. Showed large performance gap. However, not sure if this is original.
Critical weaknesses
  1. Benchmark is limited to double and single mutants. Even though training is designed for double and single mutants via pair state sampling, this doesn't mean inference is limited to those. If the model can only be used for double mutants it significantly reduces the impact of the paper, and readers will also be curious if it's just not evaluated in the paper or it is evaluated and didn't perform well.

  2. Table3: Why draw only 4 assays from ProteinGym and why only compare with weaker models? I don’t see any recent models that are higher up in ProteinGym leaderboard? I pulled up their numbers and they outperform EvoEpi with large margin.

  • RSALOR is parameter-free. It is additive baseline computed from MSA frequencies. (EvoEpi uses MSA training-time too.) also the fact that additive baseline outperforms EvoEpi on double mutant set weakens epistasis learning argument.
  • ESM3 doesn't use MSA inference-time, isn't limited to single and double mutants, yet outperforms EvoEpi. Of course ESM3 is at larger parameter size, but the paper should at least transparently state the tradeoff between performance and parameter size, rather than just excluding them and saying "EvoEpi achieves a mean Spearman correlation of.25, compared with.23 for the strongest baseline.".

And authors don't even have to inference these models themselves, they could have downloaded prediction files from ProteinGym github and computed the correlation.

AssayDouble mutantsEvoEpi, reportedBest baseline, recomputedOther strong baseline
ppluGFP215,9920.18RSALOR: 0.421ESCOTT: 0.378
amacGFP11,2600.18RSALOR: 0.431ProSST-4096: 0.400
His31,4750.15ESM3: 0.301VenusREM: 0.277
KRas abundance22,9460.40ProGen2 XL: 0.472ProSST-20: 0.438
Questions for authors
  1. Have you benchmarked beyond double mutants?
  2. Is the idea of ProteinMPNN redesign of sequence to augment the dataset beyond WT is original? I know some models use this augmentation, like ESM3 or protein design models, but is this original in this context at least? If so, I think it worth clarifying in the paper.
4 comments

W1,Q1. Scope of EvoEpi beyond double mutants.

We sincerely thank the reviewer for raising this concern. We agree that the original evaluation was limited to single and double mutations. To examine whether EvoEpi extends beyond this setting, we evaluated the same checkpoint, without additional training, on higher-order mutation subsets of PTMul and ThermoMutDB using the following inference strategy.

Scoring higher-order mutations. Retaining the notation from Section 3, let s′\mathbf{s}' denote the full mutant sequence, I=ℓ:sℓ′≠sℓ\mathcal{I}=\\{\ell:s'_\ell\ne s_\ell\\} the mutated positions, and k=∣I∣≥3k=|\mathcal{I}|\ge3. For each pair i<ji<j in I\mathcal{I}, we construct the context

\begin{cases} s_\ell, & \ell \in \\{i,j\\}, \\\\ s'_\ell, & \ell \notin \\{i,j\\}. \end{cases}$$ Thus, the other $k-2$ substitutions are present in the sequence context, while all unmutated residues remain WT. We then average the scores across the $\binom{k}{2}$ mutated pairs: $$\frac{2}{k(k-1)} \sum_{\substack{i,j\in\mathcal{I}\\\\ i < j}} g_{ij}\\!\left((s'_i,s'_j)\mid\mathbf{s}^{(ij)},\mathbf{x}\right).$$ Each pair is therefore scored in the context of the remaining substitutions. When (k=2), this formulation reduces to the original double-mutation score. **RSA calibration.** Inspired by the training-free mutational effect prediction method RSALOR [1] mentioned in the review, we additionally evaluate calibration based on relative solvent accessibility (RSA). For residue $i$, we define $$c_i = 1 - \min\left(\frac{\mathrm{RSA}_i}{100},1\right),$$ and calibrate each pair score as $$\frac{c_i+c_j}{2} \cdot g_{ij}(\mathbf{v}\mid\mathbf{s},\mathbf{x}).$$ The calibrated pair scores are then aggregated using the same higher-order scoring rule. **Higher-order mutation benchmarks and results.** We evaluate this strategy on filtered subsets of PTMul and ThermoMutDB with the following mutation-order distributions: | Number of substitutions | PTMul $\ge$ 3 | ThermoMutDB $\ge$ 3 | |---|---:|---:| | 3 | 139 | 156 | | 4 | 64 | 90 | | 5 | 16 | 24 | | 6 | 10 | 14 | | 7 | 5 | 6 | | 8 | 1 | 2 | | 9 | 3 | 4 | | 10 | 1 | 1 | | 17 | 0 | 1 | Each entry below reports global / per-protein Spearman correlation; (n) and (P) denote the numbers of variants and proteins, respectively. | Method | PTMul \$\ge\$3 | ThermoMutDB \$\ge\$3 | |---|---:|---:| | Metadata (n,P) | (239, 23) | (298, 32) | | Mutate Everything | 0.41 / 0.32 | 0.27 / 0.05 | | SPURS | 0.15 / 0.40 | 0.36 / 0.22 | | ThermoMPNN-D (pair extension) | 0.23 / 0.18 | 0.37 / 0.25 | | PottsMPNN-FT | 0.18 / 0.23 | 0.38 / 0.26 | | MSA frequency | -0.17 / -0.11 | -0.17 / 0.08 | | EvoRank | 0.35 / 0.09 | 0.50 / 0.23 | | ProteinMPNN | 0.41 / 0.35 | 0.35 / 0.30 | | ProteinMPNN+ref | 0.43 / 0.38 | 0.39 / 0.30 | | PottsMPNN | 0.26 / 0.26 | 0.34 / 0.28 | | Caliby | 0.34 / 0.22 | 0.41 / 0.28 | | RSALOR | -0.12 / 0.22 | -0.08 / 0.27 | | EvoEpi | 0.47 / 0.41 | 0.53 / 0.40 | | EvoEpi+RSA | 0.53 / 0.40 | 0.56 / 0.42 | | EvoEpi (add) | 0.36 / 0.34 | 0.45 / 0.25 | | EvoEpi+RSA (add) | 0.42 / 0.33 | 0.48 / 0.32 | For variants with three or more substitutions ($\ge$ 3), the EvoEpi models achieve the highest global and per-protein Spearman correlations on both benchmarks. EvoEpi+RSA is the only model to reach at least 0.50 global and 0.40 per-protein Spearman on both datasets. The largest gains over the competing baselines occur on ThermoMutDB, where additive RSALOR achieves 0.27 per-protein Spearman and the strongest competing ML baselines reach 0.30. EvoEpi and EvoEpi+RSA reach 0.40 and 0.42, respectively, corresponding to **48% and 56% improvements over RSALOR** and **33% and 40% improvements over the strongest competing ML baselines**. The matched ablation shows how much the epistatic terms contribute to these gains. Relative to EvoEpi (add), EvoEpi improves per-protein Spearman from 0.34 to 0.41 on PTMul (**+0.07, approximately 21% improvement**) and from 0.25 to 0.40 on ThermoMutDB (**+0.15, 60% improvement**). These gains show that the epistatic terms EvoEpi learns through self-supervised training on double mutants generalize to ranking variants with three or more substitutions on both benchmarks. These results supports that learning epistatic pairwise scoring via self-supervised learning can translate to zero-shot prediction of higher-order mutation effects. Considering residue pairs in the context of the remaining substitutions allows the same trained model to extend beyond double mutants, consistent with the broader motivation for modeling pairwise interactions in Potts models. The inference procedure follows naturally from the existing joint-scoring formulation rather than introducing an unrelated ad-hoc scoring rule.

W2. Assay selection and baseline comparisons in Table 3.

Assay selection. We thank the reviewer for highlighting the need to clarify both the assay selection and the scope of our comparisons. Our initial selection followed ThermoMPNN-D’s curation of six assays containing more than 1,000 double mutants and endpoints that could serve as proxies for thermodynamic stability. This curation excluded assays overlapping with Megascale and required either an experimental crystal structure or a high-confidence AlphaFold model.

From these six assays, we excluded two fluorescence assays, avGFP and cgreGFP, and included two non-ProteinGym assays, IRED and FLAb, to broaden the representation of biological functions. We acknowledge that omitting this selection procedure from the manuscript made the evaluation difficult to assess and could give the impression of selective reporting.

To address this concern, we expanded the evaluation to all ProteinGym assays meeting our eligibility criteria: more than 1,000 double mutants, no overlap with Megascale, and either an available crystal structure or an AlphaFold prediction with mean pLDDT $\ge$90 and at least 90% of the evaluated region having pLDDT $\ge$70. These criteria yield 12 assays, all of which are included; no additional selection was made based on phenotype or model performance. The table below reports Spearman correlations for each assay and their mean across the complete eligible set.

Method (#params)avGFPcgreGFPppluGFP2amacGFPHis3KRAS abundanceKRAS binding (K55)AAV2 capsidPAB1PSD95-PDZ3YAP1 WWGRB2 C-SH3Mean (12 assays)
Mutate Everything (657.88M)0.220.150.040.130.190.270.340.110.440.550.370.550.28
SPURS (662.54M)0.400.290.130.160.250.290.290.160.470.470.360.580.32
ThermoMPNN-D (1.78M)0.390.230.030.150.240.310.260.190.450.470.360.610.31
PottsMPNN-FT (1.71M)0.420.290.080.160.260.360.390.380.410.670.390.720.38
MSA frequency (N/A)0.120.170.040.030.150.170.180.130.330.140.250.220.16
EvoRank (19.33M)0.360.380.170.170.140.320.160.290.250.370.160.420.27
ProteinMPNN (1.66M)0.250.200.100.110.170.310.240.180.190.290.200.510.23
ProteinMPNN+ref (1.66M)0.400.300.160.160.220.250.230.290.350.520.420.580.32
PottsMPNN (1.71M)0.420.320.120.150.230.450.400.420.440.680.380.720.39
Caliby (3.93M)0.380.270.080.120.130.360.370.290.320.530.320.680.32
ESM3 (1.43B)0.430.290.060.130.300.280.560.230.650.670.440.640.39
RSALOR (N/A)0.490.560.420.430.200.240.550.590.640.660.520.620.49
EvoEpi (2.07M)0.390.320.170.170.150.400.410.500.330.610.330.650.37
EvoEpi (add) (2.07M)0.380.320.180.180.160.400.410.500.350.610.340.650.37
EvoEpi+RSA (2.07M)0.430.380.250.260.190.370.400.530.390.640.390.660.41

Scope of the generalization claim. These results do not establish that EvoEpi outperforms competing methods across all phenotypes. RSALOR achieves the highest mean correlation in this expanded comparison, and EvoEpi achieves a mean of 0.37. We therefore do not interpret these assays as evidence of a consistent advantage from joint scoring.

Our main evidence for the benefit of joint scoring comes from the stability benchmarks, including the higher-order mutation evaluations. The additional phenotype assays assess whether the learned scores remain useful beyond thermodynamic stability. We will revise the discussion accordingly and make clear that the original statement comparing mean correlations of 0.25 and 0.23 applied only to the assays and baselines evaluated in Table 3, rather than to the strongest available ProteinGym methods.

Comparison with RSALOR. We thank the reviewer for suggesting RSALOR, an important baseline missing from our original evaluation. Its strong performance on the expanded ProteinGym panel demonstrates that a training-free additive method can outperform EvoEpi on these assays. We additionally evaluated RSALOR on the stability benchmarks to examine how this comparison varies across prediction tasks. RSA-based calibration, inspired by RSALOR, also complements our approach, as demonstrated by the EvoEpi+RSA results below.

Each entry below reports global / per-protein Spearman correlation.

Method (#params)Megascale-SMegascale-DPTMul-DPTMul $\ge$3ThermoMutDB-SThermoMutDB-DThermoMutDB $\ge$3
RSALOR (N/A)0.53 / 0.580.30 / 0.240.21 / 0.10-0.12 / 0.220.36 / 0.320.12 / 0.21-0.08 / 0.27
ESM3 (1.43B)0.51 / 0.600.13 / 0.340.33 / 0.230.17 / 0.180.41 / 0.390.22 / 0.250.16 / 0.31
EvoEpi (2.07M)0.60 / 0.660.42 / 0.550.62 / 0.510.47 / 0.410.54 / 0.420.50 / 0.500.53 / 0.40
EvoEpi+RSA (2.07M)0.62 / 0.680.44 / 0.550.64 / 0.460.53 / 0.400.56 / 0.400.53 / 0.520.56 / 0.42

EvoEpi outperforms RSALOR under both metrics on all reported stability benchmarks. Together, the two evaluations show that the relative strengths of these methods depend on the phenotype: RSALOR performs strongly across the expanded ProteinGym panel, while EvoEpi performs better on the evaluated stability tasks. Incorporating RSA-based calibration into EvoEpi further improves global Spearman correlation on every reported stability benchmark, although its effect on mean per-protein correlation is mixed. These results suggest that RSA provides complementary information that can enhance EvoEpi’s predictions without changing its training procedure.

Model size and larger baselines. Our original self-supervised comparisons focused on models of comparable scale to ProteinMPNN (1.66M parameters), as noted in line 355, to reduce differences in model scale when comparing training objectives and scoring procedures. The supervised comparison also included larger models, such as Mutate Everything and SPURS, with approximately 660M parameters. Following the reviewer’s suggestion, we now include ESM3 (1.43B parameters) in both evaluations and report parameter counts alongside predictive performance to make the comparison transparent.

We agree that comparisons among similarly sized models do not replace comparisons with stronger, larger predictors. Across the 12 ProteinGym assays, ESM3 achieves a mean Spearman correlation of 0.39, compared with 0.37 for EvoEpi and 0.41 for EvoEpi+RSA, both with 2.07M parameters. EvoEpi also outperforms ESM3 under both metrics on all reported stability benchmarks. These results clarify how predictive performance varies across tasks for models of different sizes, without attributing performance differences to parameter count alone. We will incorporate these comparisons and parameter counts into the revised manuscript and explicitly scope our claims to the evaluated methods and benchmarks for greater transparency.

Q2. Originality of the sequence augmentation strategy.

We thank the reviewer for encouraging us to clarify this distinction. Using generative models to redesign protein sequences is not new,as illustrated by ESM3, and we do not claim to introduce synthetic sequence augmentation broadly.

To our knowledge, the novel contribution in this context is the integration of structure-conditioned sequence augmentation with pairwise evolutionary supervision for mutation-effect prediction. We will clarify this distinction and acknowledge related uses of inverse-folding-generated sequences, attributing novelty to this specific combination of augmentation and supervision rather than to generative augmentation in protein modeling broadly.

[1] Tsishyn, M., Hermans, P., Rooman, M. and Pucci, F., 2025. Residue conservation and solvent accessibility are (almost) all you need for predicting mutational effects in proteins. Bioinformatics, 41(6), p.btaf322.

We kindly ask the reviewer to review our rebuttal. If it addresses your concerns, we would greatly appreciate your acknowledgement and the potential reconsideration of the score. We also welcome any remaining questions or concerns and would be happy to address them during the discussion period.

Reviewer_F4bpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes to predict the effect of a double mutant (a,b)->(c,d) at positions (i,j) in the sequence by training a model which takes as input MPNN embeddings (h_i, h_j) together with learnable amino acid pair embeddings, and predicts relative empirical pair frequencies in MSAs. Specifically, the model takes as input the embeddings ((h_i,h_j,E(a,b)), (h_i,h_j,E(c,d))) and uses a Siamese MLP + merging layer to predict the output \hat{y} that is fit to match the empirical log-likelihood ratio of (a,b) vs (c,d) frequencies in real MSAs (defined as y=log(q_ij(c,d)/q_ij(a,b)) ). The paper shows that the model obtains good results on mutation prediction benchmarks.

Critical strengths
  • Method is reasonably easy to understand
  • Method seems to perform well on benchmarks
Critical weaknesses
  • The method is highly tailored towards double mutants. This is true to such an extent that to benchmark single-mutants, the proposal is to create dummy pairs using the 16 nearest residues, which is awkward. In general, the proposed solution is ad-hoc and very narrowly targetted to the double mutant case. This is the biggest weakness that leads me to find this paper less interesting.
  • Components of the model are not ablated, and natural simplifications are not explored. E.g. how much does the data augmentation \hat{s} matter? Why do we need the siamese network -- e.g. why can't one just fine-tune/add a layer to MPNN and fit with MLM on the MSA sequences? How much does the MSA matter, e.g. how does performance change with MSA depth?
Questions for authors

Besides the questions in the "Critical Weaknesses" section:

  • The model is trained to predict empirical log-likelihood ratios. For this the model uses a siamese network with a merging MLP layer. The targets y satisfy y(u, v) + y(v, w) = y(u, w) by definition, but do the model predictions satisfy \hat{y}(u, v) + \hat{y}(v, w) = \hat{y}(u, w) too? I don't see how this kind of consistency is achieved by the model. At least a comment about this would help, and why we care / do not care about this consistency.
  • When generating augmentations \hat{s}, you say the masked positions are sampled "in parallel". Does this mean they are sampled independently conditional on (s^{C}, x)? If yes, why not sample one at a time (e.g. autoregressive random order decoding)? I would guess the \hat{s} would be more "realistic" this way.
  • How did you choose how many positions to mask to create \hat{s}.
  • Can you clarify which are the learnable model parameters? Fig. 2 shows MPNN with the "fire" symbol, does that mean that you are fine-tuning some of its parameters?
  • Section A.2: "A,Bwith" => missing space.
6 comments

W1. Scope of EvoEpi beyond double mutants.

We sincerely thank the reviewer for raising this concern. We agree that the original evaluation was limited to single and double mutations. To examine whether EvoEpi extends beyond this setting, we evaluated the same checkpoint, without additional training, on higher-order mutation subsets of PTMul and ThermoMutDB using the following inference strategy.

Scoring higher-order mutations. Retaining the notation from Section 3, let s′\mathbf{s}' denote the full mutant sequence, I=ℓ:sℓ′≠sℓ\mathcal{I}=\\{\ell:s'_\ell\ne s_\ell\\} the mutated positions, and k=∣I∣≥3k=|\mathcal{I}|\ge3. For each pair i<ji<j in I\mathcal{I}, we construct the context

\begin{cases} s_\ell, & \ell \in \\{i,j\\}, \\\\ s'_\ell, & \ell \notin \\{i,j\\}. \end{cases}$$ Thus, the other $k-2$ substitutions are present in the sequence context, while all unmutated residues remain WT. We then average the scores across the $\binom{k}{2}$ mutated pairs: $$\frac{2}{k(k-1)} \sum_{\substack{i,j\in\mathcal{I}\\\\ i < j}} g_{ij}\\!\left((s'_i,s'_j)\mid\mathbf{s}^{(ij)},\mathbf{x}\right).$$ Each pair is therefore scored in the context of the remaining substitutions. When (k=2), this formulation reduces to the original double-mutation score. **RSA calibration.** Inspired by the training-free mutational effect prediction method RSALOR [1], we additionally evaluate calibration based on relative solvent accessibility (RSA). For residue $i$, we define $$c_i = 1 - \min\left(\frac{\mathrm{RSA}_i}{100},1\right),$$ and calibrate each pair score as $$\frac{c_i+c_j}{2} \cdot g_{ij}(\mathbf{v}\mid\mathbf{s},\mathbf{x}).$$ The calibrated pair scores are then aggregated using the same higher-order scoring rule. **Higher-order mutation benchmarks and results.** We evaluate this strategy on filtered subsets of PTMul and ThermoMutDB with the following mutation-order distributions: | Number of substitutions | PTMul $\ge$ 3 | ThermoMutDB $\ge$ 3 | |---|---:|---:| | 3 | 139 | 156 | | 4 | 64 | 90 | | 5 | 16 | 24 | | 6 | 10 | 14 | | 7 | 5 | 6 | | 8 | 1 | 2 | | 9 | 3 | 4 | | 10 | 1 | 1 | | 17 | 0 | 1 | Each entry below reports global / per-protein Spearman correlation; (n) and (P) denote the numbers of variants and proteins, respectively. | Method | PTMul \$\ge\$3 | ThermoMutDB \$\ge\$3 | |---|---:|---:| | Metadata (n,P) | (239, 23) | (298, 32) | | Mutate Everything | 0.41 / 0.32 | 0.27 / 0.05 | | SPURS | 0.15 / 0.40 | 0.36 / 0.22 | | ThermoMPNN-D (pair extension) | 0.23 / 0.18 | 0.37 / 0.25 | | PottsMPNN-FT | 0.18 / 0.23 | 0.38 / 0.26 | | MSA frequency | -0.17 / -0.11 | -0.17 / 0.08 | | EvoRank | 0.35 / 0.09 | 0.50 / 0.23 | | ProteinMPNN | 0.41 / 0.35 | 0.35 / 0.30 | | ProteinMPNN+ref | 0.43 / 0.38 | 0.39 / 0.30 | | PottsMPNN | 0.26 / 0.26 | 0.34 / 0.28 | | Caliby | 0.34 / 0.22 | 0.41 / 0.28 | | RSALOR | -0.12 / 0.22 | -0.08 / 0.27 | | EvoEpi | 0.47 / 0.41 | 0.53 / 0.40 | | EvoEpi+RSA | 0.53 / 0.40 | 0.56 / 0.42 | | EvoEpi (add) | 0.36 / 0.34 | 0.45 / 0.25 | | EvoEpi+RSA (add) | 0.42 / 0.33 | 0.48 / 0.32 | For variants with three or more substitutions ($\ge$ 3), the EvoEpi models achieve the highest global and per-protein Spearman correlations on both benchmarks. EvoEpi+RSA is the only model to reach at least 0.50 global and 0.40 per-protein Spearman on both datasets. The largest gains over the competing baselines occur on ThermoMutDB, where additive RSALOR achieves 0.27 per-protein Spearman and the strongest competing ML baselines reach 0.30. EvoEpi and EvoEpi+RSA reach 0.40 and 0.42, respectively, corresponding to **48% and 56% improvements over RSALOR** and **33% and 40% improvements over the strongest competing ML baselines**. The matched ablation shows how much the epistatic terms contribute to these gains. Relative to EvoEpi (add), EvoEpi improves per-protein Spearman from 0.34 to 0.41 on PTMul (**+0.07, approximately 21% improvement**) and from 0.25 to 0.40 on ThermoMutDB (**+0.15, 60% improvement**). These gains show that the epistatic terms EvoEpi learns through self-supervised training on double mutants generalize to ranking variants with three or more substitutions on both benchmarks. These results supports that learning epistatic pairwise scoring via self-supervised learning can translate to zero-shot prediction of higher-order mutation effects. Considering residue pairs in the context of the remaining substitutions allows the same trained model to extend beyond double mutants, consistent with the broader motivation for modeling pairwise interactions in Potts models. The inference procedure follows naturally from the existing joint-scoring formulation rather than introducing an unrelated ad-hoc scoring rule.

W2, Q1. Ablations and transition consistency.

Sequence augmentation. Table 6 provides an ablation of the training sequence context. Using the ProteinMPNN prior improves mean per-protein Spearman correlation over the variant without sequence augmentation (GT) by 0.22 on ThermoMutDB-D and 0.34 on PTMul-D. These results identify sequence augmentation as an important component of EvoEpi.

Relative-score regression versus likelihood modeling. The reviewer asks:

why can't one just fine-tune/add a layer to MPNN and fit with MLM on the MSA sequences?

This question is closely related to Q1 concerning transition consistency. Our model directly predicts relative scores through a Siamese architecture and a merging MLP. It does not explicitly enforce $ \hat{y}(u,v)+\hat{y}(v,w)=\hat{y}(u,w). $ By comparison, defining transition scores as differences of a common scalar log-likelihood would enforce this identity.

Our choice to regress relative scores is motivated by predictive performance. EvoRank [2] reports that, for single mutations, learning relative scores from MSA frequencies outperforms directly fitting those frequencies with a soft-cross-entropy objective. To test this alternative in our setting, we compare EvoEpi against an MPNN model trained on column-wise MSA frequencies using soft cross-entropy, without the Siamese architecture.

BenchmarkEvoEpiMPNN + column-wise MSA soft CE
Megascale-S0.60 / 0.660.58 / 0.63
Megascale-D0.42 / 0.550.38 / 0.39
PTMul-D0.62 / 0.510.60 / 0.42
PTMul $\ge$30.47 / 0.410.38 / 0.32
ThermoMutDB-S0.54 / 0.420.45 / 0.38
ThermoMutDB-D0.50 / 0.500.44 / 0.44
ThermoMutDB $\ge$30.53 / 0.400.30 / 0.31

EvoEpi achieves higher correlations across all reported benchmarks, with particularly clear gains in per-protein correlation for double and higher-order mutations. Improvements also extend to single mutations. The soft-cross-entropy baseline uses reference calibration, as do EvoEpi and ProteinMPNN+ref.

These results support the predictive value of the proposed architecture and regression objective, although they do not isolate the effect of transition consistency itself. We prioritize accurate mutation ranking in the present work; exact transition consistency is not guaranteed by our formulation.

MSA depth. We additionally evaluate training with different MSA depths. In the original configuration, joint MSA frequencies are precomputed using up to 10k10k sequences. The table below compares reduced depths with this full-depth configuration.

Benchmark1664256Full depth
Megascale-S0.60 / 0.650.62 / 0.670.63 / 0.680.60 / 0.66
Megascale-D0.31 / 0.490.33 / 0.490.35 / 0.520.42 / 0.55
PTMul-D0.58 / 0.430.62 / 0.430.62 / 0.410.62 / 0.51
PTMul $\ge$30.38 / 0.220.41 / 0.120.43 / 0.250.47 / 0.41
ThermoMutDB-S0.47 / 0.380.49 / 0.380.50 / 0.390.54 / 0.42
ThermoMutDB-D0.45 / 0.440.48 / 0.460.48 / 0.470.50 / 0.50
ThermoMutDB $\ge$30.42 / 0.340.42 / 0.350.51 / 0.320.53 / 0.40

Greater MSA depth generally improves predictive performance, particularly for double and higher-order mutations. Full depth achieves the highest per-protein correlation on every double- and higher-order benchmark, and the highest or tied-highest global correlation. The gains are especially pronounced for higher-order mutations: increasing depth from 16 to full depth raises per-protein correlation from 0.22 to 0.41 on PTMul$\ge$3 and from 0.34 to 0.40 on ThermoMutDB $\ge$3.

Although the trend is not uniformly monotonic (Intermediate depths show fluctuations in per-protein correlation, and Megascale-S performs best at depth 256), these results support the value of deeper MSAs for learning mutation effects, with the clearest benefits extending beyond single substitutions.

Q2. Parallel versus sequential sampling from ProteinMPNN.

We use parallel sampling for training efficiency. Sequential decoding could produce more “realistic” sequence contexts, but requires 16 decoder forward passes per augmented training sample when 16 positions are masked.

Empirically, parallel decoding did not reduce predictive performance in our comparison and required approximately one-third of the wall-clock training time (3.3× speedup). We attribute this to the strong structural prior provided by the ProteinMPNN encoder, which appears sufficient to produce useful augmented contexts even when the 16 positions are sampled in parallel.

Q3. Choice of the number of masked positions.

We selected the number of masked positions, ∣C∣|\mathcal{C}|, based on validation loss and training cost. Increasing ∣C∣|\mathcal{C}| increases the number of supervised residue pairs quadratically, as (∣C∣2)\binom{|\mathcal{C}|}{2}. We chose ∣C∣=16|\mathcal{C}|=16 because it achieved sufficiently low validation loss while keeping training costs manageable within our computational budget.

Q4. Trainable parameters.

The MPNN encoder and decoder are initialized from the released ProteinMPNN weights. We add the Siamese scoring architecture and merging network, and train all parameters, including those initialized from ProteinMPNN. The fire symbol in Figure 2 indicates that the scoring model’s MPNN is fine-tuned. The separate ProteinMPNN prior used to generate augmented sequence contexts remains frozen.

Q5. Typographical correction.

Thank you for pointing this out. We will correct “A,Bwith” to “A, B with” in Section A.2.

[1] Tsishyn, M., Hermans, P., Rooman, M. and Pucci, F., 2025. Residue conservation and solvent accessibility are (almost) all you need for predicting mutational effects in proteins. Bioinformatics, 41(6), p.btaf322.

[2] Gong, C., Klivans, A., Loy, J.M., Chen, T. and Diaz, D.J., 2024, July. Evolution-inspired loss functions for protein representation learning. In Forty-first International Conference on Machine Learning.

Reviewer_oLsZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EvoEpi, a self-supervised learning method for predicting the stability effects of double mutations using protein structures. The method uses pairwise amino acid frequencies from MSAs as supervision to fine-tune a pretrained ProteinMPNN model, without requiring experimental ddG labels. The authors evaluate EvoEpi on ThermoMutDB, Megascale, and PTMul, showing improvements over other self-supervised methods and outperforming supervised methods on some datasets. The authors also compare additive and direct predictions to demonstrate the benefits of modeling double mutations jointly.

Critical strengths
  1. The paper proposes a self-supervised learning method using pairwise amino acid frequencies from MSAs. It does not require experimental ddG labels and combines sequence and structural information to predict double-mutation effects.

  2. The experiments cover multiple stability datasets and other protein properties. EvoEpi outperforms other self-supervised methods on all three double-mutation benchmarks and achieves better results than supervised methods on some datasets.

Critical weaknesses
  1. The current experiments do not provide enough evidence that EvoEpi actually learns epistatic effects. In Table 1, EvoEpi does not outperform the best supervised methods on Megascale-D, and the improvements over its additive version are relatively small in global Spearman correlation. The improvement may mainly come from better single-mutation prediction rather than modeling epistasis.

  2. The comparison with additive baselines is incomplete. Table 1 only reports additive results for EvoEpi, but not for other methods. This makes it difficult to determine whether the improvement comes from the proposed pairwise modeling or simply from better single-mutation scoring.

Questions for authors
  1. The main motivation of this paper is to model epistatic effects. However, I think the current experiments are not sufficient to support this claim. In Table 1, EvoEpi is not the best-performing method on Megascale-D. On the other two datasets, the improvements over the additive version are relatively small, especially in global Spearman correlation. This makes me wonder whether the model actually learns epistasis or just learns better single-mutation information. Could the authors provide a more direct evaluation of epistatic effects, rather than only evaluating double-mutation prediction?

  2. I think the paper should also include additive results for the other baseline models. Currently, Table 1 only reports additive and non-additive results for EvoEpi. Without similar comparisons for other methods, it is difficult to tell how much of the improvement comes from modeling epistasis rather than better single-mutation prediction.

3 comments

We thank the reviewer for recognizing the strengths of our approach and for the thoughtful comments. We address the concerns below.

1. Performance on Megascale-D and modest gains over EvoEpi (add) (W1, Q1).

Supervised training on Megascale. The supervised baselines perform strongly on Megascale-D but less well on the other stability benchmarks, PTMul-D and ThermoMutDB-D. One likely explanation is that these methods were trained or fine-tuned on labeled stability measurements from Megascale, as described in lines 350–351. Training on the designated training split does not itself constitute test-set leakage, but it gives these methods access to experimental labels from the same data source as the evaluation benchmark. This is exemplified by their stronger performance on Megascale-D compared to their weaker performance on PTMul-D and ThermoMutDB-D.

Global versus per-protein Spearman correlation. The protein-engineering task in Section 2.1 is to prioritize variants for experimental validation for a given target protein to improve a particular property. Per-protein Spearman correlation directly measures how well a model ranks variants of that protein. Pooled Spearman depends on how variants from different proteins are ordered relative to one another, which is not helpful when it comes to identifying beneficial mutations during a protein engineering campaign. Furthermore, this metric is often inflated when certain proteins in the dataset are easier than others. For example, a model can improve its pooled correlation simply by shifting all predictions for one protein upward, even though it would recommend exactly the same variants for testing. Conversely, a model can improve the ranking within a protein with little change in pooled correlation if the ordering across proteins stays largely the same.

On the metric aligned with the experimental decision, EvoEpi improves over its additive counterpart by 0.07–0.13 Spearman units ( 16-34% improvement over additive). The smaller gap in pooled correlation therefore provides limited evidence about how much the epistatic terms improve variant prioritization within a target. Computing correlations within individual assays before aggregation also follows established practice in protein fitness benchmarking, including ProteinGym [1].

2. Comparison with additive baselines (W2, Q2).

We agree that the original Table 1 did not provide a sufficiently complete comparison of additive and joint scoring across methods. Our original evaluation reported each baseline using its standard scoring procedure. To address the reviewer’s concern, we now provide additive counterparts for both supervised and self-supervised methods where applicable.

Each entry below reports global / per-protein Spearman correlation.

MethodMegascale-DPTMul-DThermoMutDB-D
Mutate Everything—0.38 / 0.200.35 / 0.37
Mutate Everything (add)—0.41 / 0.340.33 / 0.37
SPURS0.57 / 0.620.50 / 0.290.38 / 0.40
SPURS (add)0.57 / 0.620.51 / 0.240.38 / 0.38
ThermoMPNN-D0.55 / 0.620.58 / 0.440.42 / 0.30
ThermoMPNN-D (add)0.54 / 0.600.58 / 0.420.43 / 0.34
PottsMPNN-FT0.50 / 0.590.59 / 0.370.44 / 0.40
PottsMPNN-FT (add)0.50 / 0.540.59 / 0.400.44 / 0.40
MSA frequency0.08 / 0.110.08 / 0.120.05 / 0.25
MSA frequency (add)0.07 / 0.150.11 / 0.020.04 / 0.17
EvoRank0.20 / 0.210.47 / 0.270.37 / 0.13
ProteinMPNN0.23 / 0.270.58 / 0.290.40 / 0.25
ProteinMPNN+ref0.42 / 0.430.60 / 0.360.41 / 0.34
ProteinMPNN (sequential)0.24 / 0.290.59 / 0.260.40 / 0.22
ProteinMPNN+ref (sequential)0.43 / 0.450.62 / 0.330.43 / 0.37
PottsMPNN0.29 / 0.360.56 / 0.310.41 / 0.28
PottsMPNN (add)0.30 / 0.360.57 / 0.280.42 / 0.31
Caliby0.19 / 0.300.53 / 0.230.39 / 0.23
Caliby (add)0.19 / 0.290.53 / 0.220.39 / 0.24
EvoEpi (epi)0.42 / 0.550.62 / 0.510.50 / 0.50
EvoEpi (add)0.40 / 0.440.60 / 0.380.46 / 0.43

EvoRank and ProteinMPNN already use additive scoring in our evaluation, so separate additive rows are unnecessary. We additionally report sequential-scoring variants of ProteinMPNN, with and without reference calibration. These variants sum two scores, with the second prediction conditioned on the first decoded residue.

Results. For most evaluated baselines, direct double-mutant scores yield correlations close to those of their additive counterparts, with gains that vary across datasets and sometimes reverse. For example, PottsMPNN-FT improves mean per-protein correlation over its additive counterpart by 0.05 on Megascale-D, but decreases it by 0.03 on PTMul-D and leaves it unchanged on ThermoMutDB-D. ThermoMPNN-D gains only 0.02 on Megascale-D and PTMul-D, while losing 0.04 on ThermoMutDB-D. By comparison, EvoEpi’s joint scoring consistently improves mean per-protein correlation across all three benchmarks. These comparisons strengthen the evidence that joint scoring contributes to EvoEpi’s predictive performance beyond summing its single-substitution scores.

[1] Notin, P., Kollasch, A., Ritter, D., Van Niekerk, L., Paul, S., Spinner, H., Rollins, N., Shaw, A., Orenbuch, R., Weitzman, R. and Frazer, J., 2023. Proteingym: Large-scale benchmarks for protein fitness prediction and design. Advances in neural information processing systems, 36, pp.64331-64379.

We kindly ask the reviewer to review our rebuttal. If it addresses your concerns, we would greatly appreciate your acknowledgement and the potential reconsideration of the score. We also welcome any remaining questions or concerns and would be happy to address them during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.