acceptodds
Under review as a conference paper at ICLR 2027

Electro-Prot: Learning Protein Representations from Electrostatics

Abstract

Protein Language Models (PLMs) trained on sequence, as well as structure-aware variants that integrate structural information, have emerged as powerful tools to learn protein representations applicable to a wide range of downstream tasks. However, neither of these frameworks explicitly accounts for the electrostatic interactions that govern protein functions. Electrostatic potentials are the primary determinants of binding, recognition, long-range interactions, and catalysis, and serve as a cornerstone of our understanding of biophysical interactions in proteins. We investigate whether a continuous electrostatic field can serve as a pretraining target to encode general protein representations. We present Electro-Prot, a 6.7 million-parameter encoder-decoder transformer model trained to predict Poisson-Boltzmann electrostatic potentials at arbitrary points around a protein. Coordinate queries cross-attend to atoms to predict the electrostatic potential as a continuous field. Electro-Prot attains (MAE = 0.81 kT/e) on a held-out test set. On downstream tasks, our embeddings match or outperform PLMs in interaction-centric tasks such as protein-protein interfacing and mutation induced change in free binding predictions, despite utilizing a dataset that is orders of magnitude smaller. Combining Electro-Prot and PLM embeddings improves results in structure similarity, protein-protein interface, Gene Ontology, and binding site detection. suggesting that Electro-Prot learns a complementary signal not captured in evolutionary statistics. Furthermore, we investigated protein-protein binding free energy predictions, where we outperformed PLMs in mutation-induced changes in binding affinity. Our results suggest that electrostatic pretraining provides a viable, parameter- and data- efficient signal for learning representations relevant to interaction-centric tasks.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein representation learning, physics-supervised learning, electrostatics, deep Learning, transformers, downstream tasks

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TCQfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Suggest EP a pretraining transformer model based on electrostatic potential data for protein representations. They have better results on interaction centric tasks.

Critical strengths

EP is an efficient model, the results seem promising, and the electrostatic potential target is well motivated.

Critical weaknesses
  1. Because EP receives full atom coordinates and partial charges, whereas the baselines are sequence models with at most structure aware token vocabularies and no pretrained structure encoder is compared, the gains cannot be attributed to the electrostatic objective rather than to richer inputs.
  2. The paper attributes PLM performance partly to test set leakage, yet EP is pretrained on nearly the entire PDB, which likely includes the downstream test structures too.
Questions for authors
  1. How do representations from the same EP architecture and inputs perform when randomly initialized, and when pretrained with a non electrostatic self supervised objective?
  2. How does EP perform when partial charges and radii are removed from the input, so that any electrostatic information must be learned from atom identity and geometry alone?
6 comments

We thank the reviewer for recognizing the efficiency of Electro-Prot and the motivation for electrostatic pretraining. We address the concerns about controlled comparisons and pretraining overlap below.

1. Richer inputs and structure-based baselines

Because EP receives full atom coordinates and partial charges, whereas the baselines are sequence models with at most structure aware token vocabularies and no pretrained structure encoder is compared, the gains cannot be attributed to the electrostatic objective rather than to richer inputs.

We agree that the effects of input information and pretraining objective should be distinguished. We now compare Electro-Prot to an alternative approach using the same network architecture and training set with a masked reconstruction loss. Compared to this purely structural training, Electro-Prot embeddings pretrained on electrostatics outperform reconstruction on 7/8 Proteinshake tasks, SKEMPI ∆∆G prediction, and 4/5 PPB-Affinity ∆G prediction datasets. The largest gains are observed in interaction centric tasks such as BS, PPI, PFam, and SKEMPI ΔΔG\Delta\Delta G prediction. These results suggest that EP's performance on downstream tasks can be attributed to electrostatic supervision rather than the architecture's rich physical inputs like partial charges and radii The masked reconstruction objective ablation shows that electrostatic pretraining generally outperformed residue reconstruction on PPI, LBA, mutation induced ΔΔG predictions, with slightly lower overall ΔG predictions, which is driven by soley Affinity Benchmark v5.5. Showcasing that electrostatic predictions allows the models architecture to learn interaction centric tasks. This aligns with electrostatics roles in binding, recognition, and complex long-range interactions in the proteins. Electrostatic supervision also outperformed masked residue reconstruction on some evolutionary based tasks like GO and Pfam, although the benefits are much smaller, further aligning with our observed limitations of electrostatic representations contribution to those tasks.

Table 11: Electrostatic supervision versus masked-residue reconstruction on EP. Values are mean ± sample standard deviation over five probe seeds. Bold indicates the better mean in each row, and Δ = Reconstruction − EP.

Task / metricElectrostatic EPReconstruction EPΔ
ProteinShake
Enzyme class (accuracy ↑).632 ± .009.635 ± .007+0.003
GO (Fmax ↑).282 ± .003.276 ± .004-0.006
Structural class (accuracy ↑).295 ± .003.216 ± .003-0.079
Protein family (accuracy ↑).249 ± .006.185 ± .002-0.064
Structure similarity (ρ ↑).559 ± .005.493 ± .007-0.066
PPI (AUROC ↑).812 ± .001.766 ± .002-0.047
Binding site (MCC ↑).323 ± .002.253 ± .001-0.070
Ligand affinity (ρ ↑).350 ± .022.346 ± .020-0.004
SKEMPI 2.0: mutation-induced ΔΔG
Pearson r ↑.446 ± .017.315 ± .014-0.131
Spearman ρ ↑.369 ± .009.302 ± .014-0.067
MAE ↓1.348 ± .0101.449 ± .013+0.101
RMSE ↓1.889 ± .0172.050 ± .029+0.161
PPB-Affinity: ΔG MAE (kcal/mol ↓)
Overall2.174 ± .0402.158 ± .019-0.016
ATLAS1.043 ± .1641.499 ± .234+0.456
Affinity Benchmark v5.52.378 ± .1261.772 ± .120-0.605
PDBbind v20201.840 ± .0101.899 ± .030+0.059
SAbDab1.442 ± .0681.540 ± .021+0.098
SKEMPI v2.02.487 ± .3203.183 ± .097+0.695

2. Pretraining overlap with downstream test sets

The paper attributes PLM performance partly to test set leakage, yet EP is pretrained on nearly the entire PDB, which likely includes the downstream test structures too.

Thank you for raising this concern. It is true that EP is pretrained on nearly the entire RCSB database and contains significant overlap. We have revised that claim in the paper and now include a section measuring sequence similarity between EP, SaProt, and ESM-2 pretraining datasets and the downstream tasks test splits using MMseqs2 search algorithm at 90%, 85%, and 80% residue identity with a minimum of. at 90% residue identity, overall similarity is 92.04%, 93.61%, and 70.82% for EP, ESM-2 and SaProt respectively. That is, we have less overlap than our main comparison, ESM, although we acknowledge that SaProt is better controlled. We will include this analysis in the updated paper.

Table 5: Sequence similarity between downstream test sets and pretraining databases for EP, SaProt, and ESM-2. Sequences are pooled across all downstream tasks

Corpus≥90% identity≥85% identity≥80% identity
ESM-293.61%96.48%98.53%
SaProt70.82%76.13%81.14%
EP92.04%92.71%94.11%

Moreover, in the section 4.1.3 "Pretraining Efficiency on a Shared Protein Corpus" experiment, we specifically controlled for test set leakage by pre-training EP, SaProt 35M, and ESM-2 8M architectures from scratch on the union of downstream training split PDBs obtained from the ProteinShake tasks, with any test or validation set PDBs removed. Under these controls, EP outperformed ESM-2 8M and SaProt 35M on interaction centric tasks at 24 hours of pretraining, where EP beats the second-best model, SaProt 35M, on binding-site prediction, LBA, and PPI. Moreover, the performance gap previously seen in enzyme class and gene ontology prediction shrank. More training time constraint results can be found in Appendix Section E Figure 4.

Matched-corpus pretraining downstream evaluation on ProteinShake tasks after 24 hours. Values are mean ± sample standard deviation over five probe seeds. Bold indicates the highest mean in each row.

Task / metricElectro-ProtESM-2 8MSaProt 35M
Enzyme classification (accuracy ↑)0.644 ± 0.0080.576 ± 0.0110.642 ± 0.023
Gene ontology (Fmax ↑)0.295 ± 0.0030.307 ± 0.0010.381 ± 0.002
Structural classification (accuracy ↑)0.249 ± 0.0160.403 ± 0.0090.694 ± 0.002
Protein family (accuracy ↑)0.250 ± 0.0010.376 ± 0.0110.562 ± 0.003
Structure similarity (ρ ↑)0.536 ± 0.0060.433 ± 0.0110.578 ± 0.013
Binding-site prediction (MCC ↑)0.281 ± <0.0010.161 ± 0.0010.240 ± <0.001
Ligand-binding affinity (ρ ↑)0.349 ± 0.0240.283 ± 0.0540.322 ± 0.029
PPI interface prediction (mean AUROC ↑)0.800 ± 0.0030.604 ± 0.0050.712 ± 0.002

3. Random initialization and alternative objectives

How do representations from the same EP architecture and inputs perform when randomly initialized, and when pretrained with a non electrostatic self supervised objective?

Thank you for suggesting these controls. We measured 3 randomly initialized untrained EP models and compared them with electrostatically pretrained EP. EP outperforms randomly initialized controls on 7/8 Proteinshake tasks, SKEMPI 2.0 mutation-induced ∆∆G predictions, and 4/5 PPB-Affinity ∆G predictions. We also included the previously mentioned residue reconstruction ablation experiment as an alternative objective for EP's architecture and inputs in section 1.

Table 6: Random-initialization control for Electro-Prot. Three independently initialized, untrained EP models are compared with electrostatically pretrained EP, using our linear probe evaluation protocol. Each entry reports the mean ± sample standard deviation across five seeds for each initialization.

Task / metricRandomly initialized EP 1Randomly initialized EP 2Randomly initialized EP 3Pretrained EP
ProteinShake
Enzyme class (accuracy ↑).632 ± .013.629 ± .012.629 ± .010.632 ± .009
GO (Fmax ↑).259 ± .002.275 ± .006.264 ± .005.282 ± .003
Structural class (accuracy ↑).208 ± .007.205 ± .005.221 ± .004.295 ± .003
Protein family (accuracy ↑).199 ± .004.229 ± .007.199 ± .007.249 ± .006
Structure similarity (ρ ↑).456 ± .022.443 ± .011.451 ± .007.559 ± .005
PPI (AUROC ↑).574 ± .003.600 ± .006.573 ± .004.812 ± .001
Binding site (MCC ↑).242 ± .001.281 ± .0004.270 ± .001.323 ± .002
Ligand affinity (ρ ↑).365 ± .019.363 ± .006.367 ± .012.350 ± .022
SKEMPI 2.0: mutation-induced ΔΔG
Pearson r ↑.288 ± .010.274 ± .012.279 ± .011.446 ± .017
Spearman ρ ↑.296 ± .009.274 ± .009.303 ± .011.369 ± .009
MAE ↓1.512 ± .0221.527 ± .0171.509 ± .0171.348 ± .010
RMSE ↓2.134 ± .0472.126 ± .0342.167 ± .0381.889 ± .017
PPB-Affinity: ΔG MAE (kcal/mol ↓)
Overall2.225 ± .0522.090 ± .0391.952 ± .0122.174 ± .040
ATLAS1.120 ± .2831.285 ± .2811.463 ± .3451.043 ± .164
Affinity Benchmark v5.51.905 ± .1761.702 ± .1491.643 ± .2792.378 ± .126
PDBbind v20201.810 ± .0221.824 ± .0311.816 ± .0071.840 ± .010
SAbDab1.612 ± .0991.577 ± .0501.529 ± .0201.442 ± .068
SKEMPI v2.03.115 ± .1912.749 ± .2973.174 ± .2712.487 ± .320
Reviewer_BFyjOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents an architecture and model to predict electrostatic potentials given coordinates, atom types, residue types, charges, and radii. The model is a 6.7M encoder-decoder transformer with various design choices on the choice of embedding, utilizing certain protein structure such as bulk colvent of calculated potentials. It is evaluated on various held out protein datasets such as ProteinShake on downstream tasks such as structural classification and similarity.

Critical strengths
  1. The paper positions itself as one of the first to model electrostatic potential information in protein language models. This seems useful compared to other models since this governs how proteins interact.
  2. From reading the experiments, it appears that for various downstream tasks, electrostatic potential information is valuable for improved performance while also being parameter-efficient.
Critical weaknesses

None. I could not parse the specific design choices nor the actual experiments in this paper as I have no domain expertise in computational biology. I am not familiar with the downstream tasks evaluated in this paper, so it is difficult for me to provide a meaningful review.

Questions for authors

N/A

3 comments
AuthorsOpenReview ↗

Model design and downstream tasks explanation.

We thank the reviewer for the positive assessment and for recognizing the value of electrostatic information and the model’s parameter efficiency.

1. Clarity of model design and downstream tasks

None. I could not parse the specific design choices nor the actual experiments in this paper as I have no domain expertise in computational biology. I am not familiar with the downstream tasks evaluated in this paper, so it is difficult for me to provide a meaningful review.

Thank you for highlighting the need for a more accessible explanation. Our central idea is to train a protein encoder by teaching it (supervised by an existing numerical electrostatic solver, APBS) to predict the electrostatic potential surrounding a protein and then use its representations for downstream molecular property prediction tasks. The key design choices of our architecture is as follows:

  • Represent atom identity, geometry, and physical inputs. We treat each atom as a token containing its element, amino-acid identity, coordinates, partial charge, and radius. This allows EP's encoder to represent the spatial arrangement of the protein's chemical and physical environment.
  • Predict a continuous field. The decoder takes a query coordinate in space and attends to the encoded atom features to predict the query's electrostatic potential. We explicitly incorporate query to atom distance into our decoders attention mechanism through an MLP because electrostatic contributions depend on the spatial separation between atoms as dictated by coulomb's law.
  • Sample informative locations. Much of the surrounding solvent environment around a protein has near zero electrostatic potentials. Our sampling strategy purposefully includes strongly positive and negative regions alongside uniformly sampled points, giving the model examples of the protein's various electrostatic environment each time a proteins passes through training.

To evaluate EP's electrostatic representations, we freeze its embeddings so that none of its learned information can change, and measure its representations predictive power using a simple linear probe which is the only trainable parameters in the evaluation. Since our model is built off atom level representations, we mean pool its atomic features into the residue or whole protein representations depending on the downstream task. The tasks we evaluate on downstream are explained below:

TaskWhat is predicted?
Enzyme classification (EC)The enzyme’s broad reaction category, given by its Enzyme Commission number.
Gene Ontology prediction (GO)Molecular function labels assigned to a protein.
Structural classification (SC)The protein’s family in the SCOP hierarchy, which groups proteins by structural and evolutionary relationships.
Protein-family classification (PFam)A protein’s Pfam family label, based on sequence relationships.
Structure similarity (SSim)The lDDT similarity score between two protein structures.
Protein–protein interface prediction (PPI)Whether each pair of residues from two interacting protein chains is labelled as a contact in the complex.
Binding-site prediction (BS)Whether each protein residue belongs to the ligand-binding pocket.
Ligand-affinity prediction (LBA)The experimentally measured affinity of a protein–ligand pair.
SKEMPI 2.0 mutation-induced (ΔΔG\Delta\Delta G)The change in protein–protein binding free energy upon mutation, in kcal/mol.
PPB-Affinity (ΔG\Delta G)The binding free energy of a protein pair.

I thank the authors for the clarification. This helps position the paper slightly better for me. I strongly recommend these details go into the revised version of the paper to make the paper more broadly accessible. I will maintain my score with the low confidence.

We thank the reviewer for their feedback and recognizing the need for broader accessibility of the paper. We have included these details in the revised paper in Section 4, Appendix B, and Table 5.

Reviewer_dS7zOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This introduces Electro-Prot that learns protein representations by predicting electrostatic potential fields computed using the Poisson–Boltzmann (PB) equation. The model incorporates atom-level structural and chemical information, including coordinates, partial charges, and atomic radii, and uses coordinate-based cross-attention with a distance-dependent pair bias to predict electrostatic potentials. The model is evaluated on ProteinShake, protein–protein binding affinity, mutation-induced binding free-energy changes, and protein–ligand binding affinity tasks.

Critical strengths
  1. This paper explores an interesting direction for protein representation learning by introducing electrostatic potential prediction as a physically motivated pretraining objective. The proposed framework provides a way to incorporate biophysical information beyond conventional sequence/structure-based pretraining.
  2. The fusion experiments provide evidence that Electro-Prot representations contain useful information complementary to existing PLMs.
Critical weaknesses
  1. The contribution of electrostatic supervision to representation learning is unclear

The central claim of this paper is that electrostatic potential prediction provides effective information for representations. However, the current experiments do not sufficiently distinguish the contribution of the electrostatic pretraining objective from input atomic structural and chemical information. Specifically, Electro-Prot directly incorporates atomic coordinates, partial charges, and radii, which are also the primary inputs used to compute the PB electrostatic potentials. Therefore, the observed improvements may arise from the richer atom-level inputs, the representation architecture, or the electrostatic supervision itself. More comprehensive ablations would help determine whether the additional physical information encoded by PB supervision provides benefits beyond additional geometric and charge-related representations.

  1. The geometric consistency of the representations is not guaranteed

Electrostatic potential is a scalar physical field that should remain consistent under joint rigid transformations of the protein structure and query coordinates. However, the proposed architecture directly encodes absolute atomic coordinates through MLPs, the complete encoder–decoder architecture does not guarantee rotation or translation invariance. Furthermore, in fusion experiments of EP + GVP, these representations are concatenated with scalar node features, potentially introducing coordinate-frame-dependent information into an otherwise geometrically equivariant architecture. GVP + EP-5 achieves a substantially lower RMSE than GVP + EP-20. The authors suggest that longer pretraining may reduce representation transferability, but the underlying mechanism remains unexplored. Another one possible explanation is that extended pretraining increases reliance on coordinate-dependent features.

Questions for authors
  1. Would the same architecture trained on a simpler charge-based potential or geometric field produce comparable representations?
  2. How stable are the learned representations under rigid transformations?

The paper presents a promising direction for incorporating biophysical supervision into protein representation learning. Although the paper still has some limitations, and my initial assessment leans toward the rejection side of borderline, I would be inclined to accept it if the authors could adequately address the key concerns raised above in weakness section and this section.

6 comments

We thank the reviewer for the thoughtful feedback and for recognizing the potential of electrostatic pretraining and its complementarity to PLMs. We address the concerns about the pretraining objective and geometric consistency below. A common thread in the feedback was for a more robust controls to isolate the benefit of training on electrostatic potentials. We now compare to a masked recontruction objective trained using the same neural network architecture and training set. We also report on the sensitivity of the embeddings to geometric transformations of the input and perform further ablations on the input descriptors.

1. Contribution of electrostatic supervision

The contribution of electrostatic supervision to representation learning is unclear

The central claim of this paper is that electrostatic potential prediction provides effective information for representations. However, the current experiments do not sufficiently distinguish the contribution of the electrostatic pretraining objective from input atomic structural and chemical information. Specifically, Electro-Prot directly incorporates atomic coordinates, partial charges, and radii, which are also the primary inputs used to compute the PB electrostatic potentials. Therefore, the observed improvements may arise from the richer atom-level inputs, the representation architecture, or the electrostatic supervision itself. More comprehensive ablations would help determine whether the additional physical information encoded by PB supervision provides benefits beyond additional geometric and charge-related representations.

We agree that distinguishing the contribution of electrostatic supervision from that of the atom-level inputs and architecture is important. We now compare Electro-Prot to an alternative approach using the same network architecture and training set with a masked reconstruction loss and an experiment where we removal physical electrostatic inputs that help EP's ability to predict PB electrostatic potentials.

Ablation 1: To further isolate the contribution of electrostatics to EP's protein representations, we compare the electrostatic pretraining objective with masked residue reconstruction pretraining under the same architecture, inputs pretraining set, and augmentations. Electro-Prot's architecture supervised on electrostatic predictions outperform residue reconstruction on 7/8 Proteinshake tasks, SKEMPI ∆∆G predictions, and 4/5 PPB-Affinity ∆G prediction datasets. Furthermore, the largest gains are observed in interaction centric tasks such as BS, PPI, SSim, and SKEMPI ΔΔG\Delta\Delta G prediction.

Table 11: Electrostatic supervision versus masked-residue reconstruction on EP. Values are mean ± sample standard deviation over five probe seeds. Bold indicates the better mean in each row.

Task / metricElectrostatic EPReconstruction EPΔ
ProteinShake
Enzyme class (accuracy ↑).632 ± .009.635 ± .007+0.003
GO (Fmax ↑).282 ± .003.276 ± .004-0.006
Structural class (accuracy ↑).295 ± .003.216 ± .003-0.079
Protein family (accuracy ↑).249 ± .006.185 ± .002-0.064
Structure similarity (ρ ↑).559 ± .005.493 ± .007-0.066
PPI (AUROC ↑).812 ± .001.766 ± .002-0.047
Binding site (MCC ↑).323 ± .002.253 ± .001-0.070
Ligand affinity (ρ ↑).350 ± .022.346 ± .020-0.004
SKEMPI 2.0: mutation-induced ΔΔG
Pearson r ↑.446 ± .017.315 ± .014-0.131
Spearman ρ ↑.369 ± .009.302 ± .014-0.067
MAE ↓1.348 ± .0101.449 ± .013+0.101
RMSE ↓1.889 ± .0172.050 ± .029+0.161
PPB-Affinity: ΔG MAE (kcal/mol ↓)
Overall2.174 ± .0402.158 ± .019-0.016
ATLAS1.043 ± .1641.499 ± .234+0.456
Affinity Benchmark v5.52.378 ± .1261.772 ± .120-0.605
PDBbind v20201.840 ± .0101.899 ± .030+0.059
SAbDab1.442 ± .0681.540 ± .021+0.098
SKEMPI v2.02.487 ± .3203.183 ± .097+0.695

1. Contribution of electrostatic supervision

The contribution of electrostatic supervision to representation learning is unclear

The central claim of this paper is that electrostatic potential prediction provides effective information for representations. However, the current experiments do not sufficiently distinguish the contribution of the electrostatic pretraining objective from input atomic structural and chemical information. Specifically, Electro-Prot directly incorporates atomic coordinates, partial charges, and radii, which are also the primary inputs used to compute the PB electrostatic potentials. Therefore, the observed improvements may arise from the richer atom-level inputs, the representation architecture, or the electrostatic supervision itself. More comprehensive ablations would help determine whether the additional physical information encoded by PB supervision provides benefits beyond additional geometric and charge-related representations.

Ablation 2: To assess the contribution of the explicit physical inputs EP is trained on and the effects it has on EP's ability to learn electrostatic information, we conduct an ablation where we zero partial charges and atomic radii during pretraining and downstream evaluation, while we retain all other EP inputs, original architecture, pre-training configurations, and data augmentations. Without charges/radii, EP still learns to predict electrostatic potential, achieving a Pearson correlation of 0.9620.962 on the validation set.

Unfortunately, due to limited computational resources, we do not yet have results for 20 epochs of training. Results for 5 epochs are shown in Table 12. Downstream metrics are mixed with most performances differences being minimal. We observe that removing partial charge and radius information results in slower initial training, but both models have largely converged to the same loss by 5 epochs.

Table 12: Charge and radius input ablation. EP and no charge/radii EP checkpoints are evaluated after five pretraining epochs using the established probe settings. EP’s model was retrieved from our 5 epoch pretraining checkpoint, while the zero charges/radii EP was pretraind from scratch. Values are mean ± sample standard deviation across five probe seeds. Bold indicates the better mean in each row

Task / metricEPEP without charges/radii
ProteinShake
EC (Acc ↑).652 ± .016.626 ± .012
GO (Fmax ↑).322 ± .006.298 ± .004
SC (Acc ↑).274 ± .008.312 ± .006
PFam (Acc ↑).283 ± .004.254 ± .007
SSim (ρ ↑).549 ± .005.529 ± .003
PPI (AUROC ↑).808 ± .001.794 ± .003
BS (MCC ↑).356 ± .002.369 ± .001
LBA (ρ ↑).352 ± .019.334 ± .018
SKEMPI 2.0: mutation-induced ΔΔG
r ↑.380 ± .028.472 ± .014
ρ ↑.357 ± .012.405 ± .009
MAE ↓1.411 ± .0171.332 ± .014
RMSE ↓2.005 ± .0271.868 ± .021
PPB-Affinity: ΔG MAE (kcal/mol ↓)
Overall2.097 ± .0542.108 ± .044
ATLAS1.410 ± .3471.481 ± .529
Affinity Benchmark v5.52.121 ± .0912.261 ± .107
PDBbind v20201.824 ± .0131.790 ± .015
SAbDab1.493 ± .0581.566 ± .055
SKEMPI v2.02.685 ± .2032.840 ± .454

Overall, these ablations supports our claims that pretraining on electrostatic potential predictions is what drives Electro-Prot to learn protein representations rather than the rich atom level inputs it receives. The masked reconstruction objective ablation shows that electrostatic pretraining generally outperformed residue reconstruction on PPI, LBA, overall ΔG predictions, and mutation induced ΔΔG predictions, showcasing that electrostatic predictions allows the models architecture to learn interaction-centric tasks. This aligns with the role of electrostatics in binding, recognition, and complex long-range interactions in proteins. Electrostatic supervision also outperformed residue reconstruction on some evolutionary based tasks like GO and Pfam, although the benefits are much smaller, further aligning with our observed limitations of electrostatic representations contribution to those tasks. On the other hand, partial charges and radii provided EP with informative physical inputs, but are not necessary for learning predictive electrostatic representations, and their removal improved some downstream tasks such as mutation-induced ΔΔG and binding site detection. Overall, these results support electrostatic prediction as a useful protein representation learning objective. We will include this analysis in the updated paper.

2. Rotation and translation consistency

The geometric consistency of the representations is not guaranteed

Electrostatic potential is a scalar physical field that should remain consistent under joint rigid transformations of the protein structure and query coordinates. However, the proposed architecture directly encodes absolute atomic coordinates through MLPs, the complete encoder–decoder architecture does not guarantee rotation or translation invariance.

Thank you for raising this point. We adopted a "learned equivariance" approach where the model is trained with randomly rotated structures. Inputs always are centered to address translation concerns.

Electrostatic Potential Prediction Sensitivity Experiment:

For quantifying the sensitivity of electrostatic potential predictions on rotations, we evaluated the frozen EP model on 8913 test proteins. For the identity (untransformed proteins) and ten joint rotations of atoms and queries, we used the same 2048 uniformly sampled finite-grid query identities per protein and retained their original APBS labels (i.e. we ran APBS query only once for the untransformed query coordinates since potential is invariant).

In this experiment and the table below, we looked at the following metrics:

  • Mean absolute rotation-induced EP prediction vs. identity which measures the average absolute difference between transformed and identity EP predictions at corresponding query points.
  • ΔMAE against APBS, transformed minus identity (kT/e): This measures the transformed prediction’s MAE against the same APBS reference minus the identity prediction’s MAE, with positive values indicating worse agreement and negative values indicating improvement.
  • CIs: Brackets show 95% bootstrap confidence intervals for the reported means, obtained from 2,000 whole-protein resamples after averaging transformations within each protein; they describe uncertainty in the means.
OrientationMean absolute rotation-induced EP prediction vs identity (kT/e)ΔMAE against APBS: transformed minus identity (kT/e)
Rotation0.25056 [0.24695, 0.25435]+0.00525 [0.00301, 0.00733]
Translation1.75×10−61.75\times10^{-6} [1.72×10−61.72\times10^{-6}, 1.78×10−61.78\times10^{-6}]+5.81×10−10+5.81\times10^{-10} [−5.94×10−9-5.94\times10^{-9}, 6.57×10−96.57\times10^{-9}]

The mean rotation-induced change of 0.250560.25056 kT/e is significantly smaller than the model's mean error against APBS 1.636151.63615 kT/e (identity baseline). Additionally, rotation increased the mean APBS error by a negligible 0.005250.00525 kT/e. For context, rotating a protein and recalculating the electrostatics with APBS has a mean error of 0.05 kT/e (see below).

APBS Numerical Calculation Sensitivity:

For this experiment, we selected protein 5ooo, Rift Valley fever virus NSs core domain, since this was the median ranked protein by atom count (5058 atoms). For identity and ten rotations, we solved for the electrostatic potential with APBS (0.5Å grid resolution). The same 2048 uniformly sampled queries were rotated with the protein and each grid spacing was compared with its own identity solution. We averaged the absolute differences over queries and rotations.

APBS grid spacingUniform-query mean orientation disagreement (kT/e)
0.5 Å0.05322

4. Effect of pretraining duration

GVP + EP-5 achieves a substantially lower RMSE than GVP + EP-20. The authors suggest that longer pretraining may reduce representation transferability, but the underlying mechanism remains unexplored. Another one possible explanation is that extended pretraining increases reliance on coordinate-dependent features.

Furthermore, in fusion experiments of EP + GVP, these representations are concatenated with scalar node features, potentially introducing coordinate-frame-dependent information into an otherwise geometrically equivariant architecture.

We emphasize that we randomly rotate input structures, so it is impossible to overfit to coordinate-frame dependent features - no set of coordinates is seen more than once during training. This does not prevent the model from overfitting to derived, rotationaly invariant features, such as the distances used in the pair bias.

While EP-5 showed improvement in ligand binding affinity (LBA) prediction, as we now show in our random initialization experiment (see section 3 in response to reviewer TCQf), LBA performance is largely driven by ligand identity rather than protein representation. We will down-weight our reporting on this task in the main paper.

Comments on the paper

2 comments

Hi reviewers,

Thank you for your reviewing service. The authors have provided fairly extensive rebuttals. Please confirm that you have read them, state which of your concerns are or are not addressed, and whether that changes your recommendations.

We thank all the reviewers for their feedback on our paper. We greatly appreciate all the suggested experiments and controls, and have revised the paper to include them.

Several reviewers have highlighted the need for more robust controls to isolate the benefit of training on electrostatic potentials from our rich atom level inputs and architecture on downstream tasks. To address this we conducted two ablations: Pre-training EP on an alternative masked residue reconstruction objective under the same architecture and rich atom level inputs, and pretraining EP without incorporating partial charges and radii as physical inputs. Electrostatic pretraining outperformed masked residue reconstruction on 12 of 15 tasks, with the most significant performance gains seen on interaction based tasks such as PPI, binding site, structure similarity, and mutation induced changes in binding affinity predictions. Without charges/radii, EP still learns to predict electrostatic potential (r=0.962r = 0.962), and the rich physical inputs were not necessary for learning predictive electrostatic representations as downstream tasks only slightly degraded, with some tasks improving such as mutation-induced ΔΔG and binding site detection. Overall, these results support electrostatic prediction as a useful protein representation learning objective, and aligns with the role of electrostatics in governing the interaction of proteins. We also investigated how sensitive EP is to protein orientation. Rotating the proteins slightly changed EP's predicted electrostatic potential, but had a a negligible effect on accuracy against APBS, increasing MAE by only 0.005 kT/e0.005\,kT/e. Rotation also had a very little effect on downstream PPI performance (AUROC of 0.805870.80587 before rotation vs 0.804660.80466 after rotation), indicating that EP has learned equivariant representations. We have also expanded our downstream task explanations to make the paper more broadly accessible to the wider ML community.

A summary of the main revisions of the paper can be found below:

  • Masked residue reconstruction with EP's architecture, Section 5.2 and Appendix G.2
  • EP perform when partial charges and radii are removed from the input, Section 5.3, Appendix G.3
  • Electrostatic Potential Prediction and APBS Numerical Sensitivity to Rotations and Translations, Section 4.1, Appendix J.1
  • Downstream tasks explanations for broader accessibility, Section 4, Appendix B, Table 5
  • Pretraining and downstream tasks test split overlap, Appendix B.6
  • Random initialization of EP Representations on downstream tasks, Section 4.1.6, Appendix C
  • Sensitivity of EP Representations to Rotations for Downstream Prediction Tasks (PPI), Appendix J.2

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.