acceptodds
Withdrawn from ICLR 2027

ProtSurf: Learning Transferable Protein Surface Representations from Large-Scale Protein Interfaces

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Protein sequence and structure have emerged as powerful reusable modalities for protein representation learning. In contrast, protein surfaces—which mediate protein-protein interactions—remain underexplored as transferable representations. We introduce ProtSurf, a hierarchical self-supervised framework that formulates protein surface learning as a scalable representation learning problem and learns residue-aligned surface representations from large-scale protein interfaces. ProtSurf models protein surfaces at local and global scales and investigates how protein surface representations benefit from increasing model capacity and pretraining data diversity. We pretrain ProtSurf on approximately 200,000 experimentally determined RCSB protein structures and 1.6 million domain–domain interfaces derived from AlphaFold-predicted structures. We evaluate ProtSurf residue embeddings for binding affinity prediction and achieve strong performance across multiple benchmarks. Beyond affinity prediction, ProtSurf provides, to our knowledge, the first transferable residue-level surface representation learned at scale, establishing protein surfaces as a representation modality alongside sequence and structure.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Protein representation learning, Self-supervised learning, Protein-protein interactions, Geometric deep learning, Protein structure, Binding affinity prediction, Multimodal protein modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nnwhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

They propose to learn residue-aligned surface embeddings with a self-supervised reconstruction framework. A local graph is formed by 32 surface triangles around each surface residue (with Masif features + position wrt to backbone), with adjacency given by relative positions. This surface graph is encoded as discrete tokens, a protein-level message passing happens, and the network aims to reconstruct the 32 input features. To do so, a permutation loss is used. Then, those surface tokens enrich GearBind for protein mutation stability prediction. They obtain sota results on this task. They vary model sizes from 10M to 650M, and pretrain on ~200k RCSB structures, ~440k chains extracted from complexes and 1.6M AFDB domain–domain interfaces, showing interesting scaling laws.

Critical strengths

S1. Strong SOTA results on an important task.

S2. Arguably the first large-scale pretraining of a surface network.

Critical weaknesses

W1. The scaling effect are clear but seem to dominate architectural effects. On a 35M model, with PDB data, is there a gain in including surfaces compared to GearBind+PT? What is the GearBind+PT baseline, how is Gearbind pretrained when it's not with ProtSurf embeddings? Current results suggests that scaling pre-training across model dimension and data size strongly helps for mutation stability prediction, but I am not fully convinced that surface was necessary.

W2. Comparison to existing pre-training approaches is sometimes faulty

  • Pre-training Sequence, Structure, and Surface Features for Comprehensive Protein Representation Learning propose a surface-informed pretraining technique. It represents an important comparison.
  • AtomSurf is presented as a supervised method in Table 1, and as a pre-trained method in Table 2. To my knowledge, it is not pretrained.
  • Other pre-trained structural tokenizers exist (ESM3, SLAE, ISM...), but do not include surfaces. This would be an interesting ablation for W1, and at least worth mentioning in related work.
Questions for authors

Q1. Is there a reason to not use a non-equivariant point transformer in internal coordinates (using 3 points to define a backbone frame instead of just 2: distance and angle)?

Q2. Could you report on the computational aspects of you work? You mention that adding small tokens has limited impact on GearBind parameter count, but you still need to create a surface, patch it, and run you model on each residue. I think readers will want to know how costly this is.

Q3. Following Q2, could you use faster meshing methods? dMasif and AlphaSurf are compatible with on-the-fly processing, which would favor adoption for your method. The definition of a patch was confusing to me "As not all of the patches will be mapped to the residue (as per MaSIF calculation), we then categorized the patches into three groups: (1) Core — patches where all three vertices map to atoms from residue". There seems to be a confusion between patches and triangles here. If you only need points at the surface, dMasif is the most established tool. If you need meshes, AlphaSurf would be more adapted.

Reviewer_oeWYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ProtSurf, a self-supervised framework for learning residue-aligned protein surface representations. It encodes local geometric and physicochemical surface features and uses protein-level context to reconstruct surface graphs, with finite scalar quantization providing discrete representations. Pretraining uses experimental structures and predicted domain–domain interfaces. Frozen embeddings are incorporated into GearBind for binding-affinity prediction, with experiments examining model scaling and architectural choices.

Critical strengths
  1. A useful design for learning reusable surface representations. The authors map local geometric and physicochemical surface features to individual residues, making the representations compatible with other residue-indexed protein models. The reconstruction objective combines local surface properties and connectivity with protein-level context. Together with pretraining on experimental structures and predicted domain interfaces, this provides a practical approach to learning surface representations from large collections of unlabeled structures.
  2. Promising results supported by scaling experiments and ablations. ProtSurf reports strong performance on SKEMPI and improvements on the evaluated antibody benchmarks while keeping the surface encoder frozen. The model-scaling experiments maintain a fixed embedding dimension and report results across multiple seeds. The ablations also provide useful evidence for the contributions of residue anchoring, global context during reconstruction, and surface connectivity reconstruction.
Critical weaknesses
  1. The breadth of the transfer evaluation is limited. The experiments demonstrate that frozen surface embeddings are useful for downstream affinity prediction, but all evaluations use GearBind and closely related prediction settings. Their usefulness across different downstream architectures, tasks, and previously unseen protein families remains insufficiently established. The external antibody evaluations cover few scaffolds, with modest improvements on HER2 and a small set of validated beneficial mutations in the SARS-CoV-2 case study. The RCSB pretraining source corpus also contains the structures represented in SKEMPI. The authors acknowledge this overlap and do not use affinity labels during pretraining. As they explicitly note, this setting does not test strict generalization to structures unseen during pretraining. Evaluations on unseen protein families and additional downstream tasks would strengthen the case for broader reuse of these representations.
  2. The comparisons do not sufficiently establish what drives the performance improvement. Adding ProtSurf embeddings introduces both surface information and a pretrained representation, but the evaluation does not clearly separate these contributions. The architectural ablations are informative, yet they do not establish how much large-scale pretraining contributes beyond providing the downstream predictor with the same surface descriptors through a simpler encoder. Moreover, the main comparisons use published baseline results that may follow different evaluation protocols, limiting their use as controlled evidence for the reported gains. The authors need matched comparisons to establish the benefit of the learned representations over simpler ways of incorporating surface information.
Questions for authors
  1. Could the authors report the overlap between the actual pretraining data used for the evaluated checkpoints and the downstream benchmarks, including related protein families? Could they evaluate a structure- or family-disjoint setting and an additional task, such as protein–protein interface prediction, to better establish transferability?
  2. Were the GearBind+PT results in Table 2 rerun using the same data processing, training/validation/test partitions, and checkpoint-selection procedure as ProtSurf? Section 5 describes the baselines as published results; could the authors provide a controlled comparison under a shared protocol?
  3. Could the authors compare pretrained ProtSurf against a randomly initialized frozen encoder and a lightweight encoder using the same surface descriptors, while keeping the downstream GearBind setup fixed? This would help distinguish the benefit of pretraining from the benefit of adding surface information.
  4. Which representation is extracted for downstream prediction: the local encoder output, the quantized embedding, or the output of the global module? Appendix F.1 places the global module after quantization, whereas Figure 1’s tokenization pathway omits it. Could the authors clarify whether global contextualization runs during downstream feature extraction or only supplies supervision during pretraining?
Reviewer_PAtEOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes ProtSurf, a two-level self-supervised model that produces one embedding per surface-exposed residue of a protein. At the local level, each residue owns a graph of N=32 sampled surface patches carrying seven MaSIF-style descriptors (electrostatic potential, hydrophobicity, hydrogen-bond indicators, shape index, distance-dependent curvature, plus two added geometric scalars: patch-centroid-to-Cα distance and the centroid–Cα–Cβ angle). A graph Transformer encodes this local graph; its virtual-node output is quantized with FSQ; a second graph Transformer contextualizes the whole set of residue codes over a protein-level Cα graph; a permuter plus a local decoder then reconstruct the original patch graph's node features and adjacency. The loss is reconstruction of the input local graph plus an entropy penalty on the soft permutation. Pretraining uses ~200k RCSB entries (plus ~440k partner-removed chain views) and 1.6M TED-derived AlphaFold domain–domain interfaces. Five model sizes (10M–650M) are trained for a fixed 100k steps. Downstream, the encoder is frozen, its 128-d residue embeddings are concatenated into GearBind, and ΔΔG is predicted on SKEMPI v2, HER2, and a five-mutation SARS-CoV-2 ranking case study. The claim is that the protein surface should be treated as a reusable representation modality alongside sequence and structure.

Critical strengths

S1. The scaling ladder is internally clean, and the negative results are reported honestly. All five sizes share corpus, sampling distribution, batch size, optimizer, LR, and step count, differing only in the dimensions of Table 7; results are given as mean ± std over 15 runs (Table 9). The paper also reports, against its own interest, that removing FSQ improves Pearson by 0.006 and states that quantization is a representational format rather than a source of accuracy. That is the kind of result most submissions bury.

S2. Residue alignment is the correct interface decision, and the ablation supports the mechanism. Placing surface information on the residue index is what makes it composable with SaProt/ProSST/ESM3-style models, and the ablations are directionally coherent: global contextualization +0.041, anchor features −0.062 if removed, adjacency reconstruction −0.022 if removed, embedding-dimension changes −0.015/−0.030 in both directions. The inverted-U on embedding width is a useful control against "the gain is just more dimensions".

S3. The evaluation protocol is described more carefully than is usual in this sub-area. Complex-level train/validation splits with no shared complex, checkpoint selection on validation only, 3 folds × 5 seeds, explicit exclusion of Pythia-PPI / PreFold-dG / DSSA-PPI because their splits are not comparable, and a Limitations section that volunteers the RCSB/SKEMPI overlap rather than hiding it. The overlap is a problem (W2) but the candor is real.

Critical weaknesses

W1. The central claim is not isolated from capacity: 89% of the headline margin is a scaling knob. Table 2 puts GearBind+PT at 0.6760 and "ProtSurf (650M)" at 0.7612, attributing +0.0852 to surface information being complementary to all-atom features. But Table 5 puts the 10M model at 0.6856. So the 10M scale buys +0.0096 over GearBind (11% of the headline), and the 10M→650M step buys +0.0756 (89%). HER2 tells the same story: 0.515 → 0.516 (10M) → 0.534 (650M), i.e. 95% of the reported gain is capacity. The defence in §5.1 — "GearBind has ~1.8M parameters and the 128-d embeddings add negligible parameters" — does not hold: those embeddings are produced by a 361× larger encoder. As written, the evidence supports "a 650M surface encoder yields useful features", not "surface information is complementary". There is no capacity-matched, surface-free control anywhere in the paper: no randomly initialized encoder, no equal-width non-surface feature (ESM-C / SaProt / ESM-IF), no shuffled-target control. Without one, the modality claim and the capacity story are indistinguishable, and the capacity story is the cheaper explanation.

W2. Transfer is never measured: every benchmark structure was in pretraining, and every benchmark is ΔΔG. §4 states that all 345 SKEMPI PDB entries are in the May-2024 RCSB corpus, calling this "benchmark-input overlap". The chain-extraction step (C.2) then removes partner chains and recomputes MaSIF features, explicitly exposing interface regions of those same complexes as additional views. The HER2 and SARS heads are trained on a SKEMI split. So across all three evaluations the only genuinely unseen quantity is a label vector, never a structure. "Transferable" in the title, "transfer" in §1/§5, and "the first transferable residue-level surface representation learned at scale" in the abstract are therefore unearned; a Limitations concession does not license a title. The second half of the problem is scope: all three tasks are ΔΔG on protein–protein or antibody–antigen complexes through the same small all-atom GNN head. A claim that surface is "a modality alongside sequence and structure" needs one task where surface geometry is necessary and backbone atoms insufficient.

W3. The objective is an unmasked autoencoder, undefined in the main text, with a leak in its own target. §3.3 defers the entire objective to Appendix B.5. Read literally, the full local patch graph is reconstructed from the residue code — no masking, no corruption, no target augmentation — so the only pressure is the 128-d FSQ bottleneck. This is an autoencoder over MaSIF descriptors, whereas the closest competitor (Surface-VQMAE) uses masked reconstruction; no masked, denoising, or corruption-ratio variant is reported, so the objective is inherited rather than selected. The target also contains the two anchor features, which are near-deterministic given the residue correspondence the encoder is anchored to. The −0.062 for removing anchors (F.3) is thus consistent with two incompatible readings: alignment information genuinely helps, or the target contains a free shortcut that inflates pretraining signal without teaching transferable chemistry. The present ablation cannot separate them. Relatedly, "self-supervised" here means "supervised by MSMS + APBS + hand-designed physicochemical descriptors", not by raw atoms. That should be said once, plainly.

W4. Presentation: no equations, unreadable figure, undefined core term.

  • §3 contains zero formulas, while the loss (Eq. 12: L = L_rec + λC(P̂)), the permutation relaxation (Eq. 9) and attention (Eq. 1–5) sit in Appendix B. For a paper whose contribution is a training objective, Eqs. 10–12 and the FSQ quantizer belong in the main text.
  • Figure 1 is unreadable at print size (~15 micro-labels in a ~2.5-inch panel, effective 4–5 pt) and its two panels redraw the same block diagram.
  • The objective's direction is unlabeled. "True surface features per residue" → encoder → … → "Predicted surface features per residue" is drawn left-to-right with no mark that the left box is the target. For a reconstruction objective this is normal practice, but the figure must draw the loss and show that encoder input and decoder target are the same tensor.
  • "Protein surface" is never defined. The term appears 116 times in the main text; "solvent", "triangulated" and "solvent-excluded" appear zero times; MSMS appears once, in the Limitations. The actual object (a triangulated solvent-excluded surface mesh, 9 Å geodesic patches, N=32 per residue, each patch parameterized on a local polar grid) is only in A.1. One sentence in §3.1 fixes this.
  • Minor: Table 2 and Table 8 are the same table; Table 9 duplicates Table 5; "SPD" is undefined in Figure 1; C.3 reads "1.6M million".
Questions for authors
  1. Capacity-matched control. Report GearBind + 128-d frozen features from (a) a randomly initialized 650M ProtSurf, (b) a non-surface modality at matched width (ESM-C or SaProt), (c) ProtSurf 650M with shuffled reconstruction targets. If (a)–(c) close most of the 0.6760→0.7612 gap, what remains attributable to surface pretraining rather than capacity?

  2. Structure-held-out retraining. Remove all SKEMI/HER2/SARS parent PDB entries and their chain views from pretraining, retrain 60M and 650M, and report SKEMPI, HER2 and the change in the scaling slope. Does the +0.075 capacity trend survive?

  3. Objective and target leakage. Give masked-node reconstruction at 30/50/75%, plus noise on node features, at matched compute. Separately, decompose reconstruction error into the 5 MaSIF channels vs the 2 anchor channels, and train a variant that conditions on the anchors without reconstructing them. Is the −0.062 an alignment effect or a target shortcut?

  4. The adjacent claim, cheap to answer. What fraction of residues per protein receive an embedding, and how does GearBind consume the missing indices? Then add one non-ΔΔG task (interface residue classification or protein–ligand affinity); without it the "modality alongside sequence and structure" framing is positioning, not evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.