acceptodds
Under review as a conference paper at ICLR 2027

CAN MOLECULAR-DYNAMICS SUPERVISION HELP PREDICT PROTEIN FUNCTIONS?

Abstract

Protein functions depend on conformational ensembles, but incorporating molecular-dynamics (MD) information into sequence-based prediction requires supervision that can be learned without trajectories at inference. We introduce ESMLace, an ESM-2 35M encoder trained to predict compressed MD targets: moments, wavelets, cosine coefficients, or discrete VQ-VAE tokens from 96-frame windows. We transfer the frozen encoders to mutation-effect prediction and immune-recognition benchmarks, comparing matched mean-pool and token-level readouts. Across ten ProteinGym assays, VQ supervision changes from −0.008 Spearman relative to Base ESM under mean pooling to +0.166 under a two-block token readout, giving a compression-by-readout interaction of +0.174 (95% assay bootstrap [0.083, 0.271], family-wise p = .031). VQ supervision also improves BATCAVE global peptide-activity ranking from 0.495 to 0.557 Spearman, while antibody and TCR–pMHC results vary by benchmark. Replica and matched-probe analyses identify reproducible MD signals and increased target decodability, while waveform reconstructions distinguish temporal-shape recovery from amplitude recovery. MD supervision can therefore improve sequence-based function prediction, with its benefit determined jointly by temporal representation, readout capacity, and task-label alignment.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular-Dynamics, TCR-pMHC, protein function, protein fitness prediction

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_aPt3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

Authors fine-tune a small ESM trunk on features extracted from MD. Then, they use this MD-enriched representations for several downstream tasks, and show some features do improve the performance over ESM2-35M.

Critical strengths

S1. The paper is generally easy to follow, and the experiments also report negative results on fair benchmarks

Critical weaknesses

W1. Related work is overlooked. You are not the first to think about the potential of leveraging dynamics to enhance protein function prediction. In the structure encoding realm, several papers come to mind [1-4]. But papers also exist in the LLM realm [5-6]. And several papers included structure, which represents a key baseline [7-8].

W2. Does this paper enhance SOTA ? What authors show here is "esm+dynamics > esm", but don't include comparison to sota on the tasks. They only fine-tune a small esm, but I doubt this corresponds to the strongest performance. Maybe fine-tuning ESM-C would have a better chance. If you have computational constraints, you can use PEFT.

W3. On ProteinGym, you only include 10 assays, and only report on one splitting strategy; it's hard to compare to sota from this.

[1] Encoding protein dynamic information in graph representation for functional residue identification

[2] Boosting Protein Graph Representations through Static-Dynamic Fusion

[3] Learning Dynamic Protein Representations at Scale with Distograms

[4] From Static to Dynamic Structures: Improving Binding Affinity Prediction with Graph-Based Deep Learning

[5] Protein Language Modeling beyond static folds reveals sequence-encoded flexibility

[6] ESMDynamic: A Fast and Accurate Prediction of Protein Dynamic Contact Maps from Single Sequences

[7] SaProt: Protein Language Modeling with Structure-aware Vocabulary

[8] Bilingual language model for protein sequence and structure

Questions for authors

See weaknesses.

Reviewer_afajOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Summary: The paper proposes a way to improve protein function prediction of sequence-based models (like ESM) by training the sequence encoder on molecular dynamics trajectories. ESM-2 35M encoder is trained to predict compressed MD descriptors derived from 96-ns windows and then the frozen encoders are used for downstream tasks.

Contributions:

  1. ESM-Lace: a family of ESM-2 encoders trained on compressed trajectory descriptors from MD trajectories.
  2. The authors experimented with different ways of using the tokens from the encoders and found that a two-block readout where the tokens are mixed is better than mean pooling.
Critical strengths

I don't find any specific strength for this paper. The only contribution would be the two-block readout interaction (instead of mean pooling) that seems to improve results, but is just a trick on how to use a frozen encoder for a downstream task. The underlying idea of improving sequence-based protein representations with dynamical information is well-motivated. However, the execution is not sound (see weaknesses).

Critical weaknesses

The main weakness is that features from MD of 96-ns windows do not encode the dynamics the paper claims to exploit.

The motivation of the paper is to transfer conformational dynamics information into sequence representations via MD supervision. However, training is performed on statistics derived from 96-ns trajectory windows. It is well known that proteins (also the ones present in the BioEmu dataset that the authors use) exhibit functionally relevant conformational changes on µs–ms timescales. At sub-100-ns spans, the captured signals reflect fast local fluctuations and there is not a reason to expect that statistical summaries of these short windows encode information about conformational heterogeneity relevant to binding, recognition, or mutational fitness. The mixed and negative results on the immune benchmarks, where slow dynamics are most relevant, are consistent with this fundamental limitation.

Questions for authors
  1. Can the authors provide evidence that these short-window features encode information about dynamics beyond fast local fluctuations?
  2. Can the authors show that the observed gains from training on MD features generalize to other sequence encoders?
Reviewer_ovGfOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether supervising a protein language model (PLM) encoder with compressed molecular-dynamics (MD) targets helps downstream sequence-based function prediction. "ESMLace" is an ESM-2 35M backbone jointly trained with lightweight heads that regress/classify four families of compressed descriptors from 96-frame MD windows — summary statistics, Haar wavelets, DCT coefficients, and discrete VQ-VAE tokens — over either a full descriptor set or a geometry-only subset. MD corpus: 23,922 BioEmu-derived proteins. After training, MD heads are discarded and the frozen encoder transferred — via mean pooling or a trainable two-block Transformer token-mixing readout — to ten ProteinGym assays plus four immune-recognition benchmarks.

VQ-VAE is marginally worse than Base ESM-2 under mean pooling but clearly better under the two-block readout, an interaction of +0.174. Immune results are mixed: BATCAVE improves; AbRank improves on average but with huge seed variance; TABR-BERT S1–S4 is slightly worse for VQ than base; ITRAP shows a large nominal gain that the authors themselves flag as likely confounded by component recurrence (99.57% of test TCRs seen in training) and a sub-random Base-ESM baseline.

Critical strengths

A real, statistically-supported representation×readout interaction, carefully isolated. The core ProteinGym result isn't "MD helps" or "VQ is best" — it's that VQ-style supervision hurts under mean pooling and helps under token-mixing, and the interaction itself carries the strongest signal. This is backed up by the Base-ESM readout contrast being flat (0.381→0.392) while VQ(D&D)'s is large (0.373→0.558) — arguing against "two-block readout is just generically better." This matters beyond this paper: it shows readout architecture is a first-order variable for dynamics-supervised PLMs, not an implementation detail, and papers testing only mean-pooled frozen embeddings may be mis-estimating the value of MD supervision.

Critical weaknesses
  1. Base-ESM never gets the same additional gradient exposure as the MD arms — an unaddressed continued-pretraining confound. Every arm starts from the same ESM-2 35M weights, but the MD-supervised arms are then optimized jointly with MD heads for up to 20 epochs over the MD corpus, while Base ESM-2 appears to just be the off-the-shelf checkpoint, with no matched continued-MLM-training control on the same sequences. So every MD-vs-Base contrast in the paper (Tables 3–11, the whole headline interaction) confounds "effect of the MD target" with "effect of extra domain-adaptive training on real folded/mutant sequences." Continued in-domain pretraining alone is known to shift PLM representations, so this could meaningfully inflate the apparent MD-specific benefit. This is the single assumption the paper's title question rests on, and the obvious cheap fix — continue-train ESM-2 with plain MLM, no MD head, same budget — is never reported.

  2. Single MD-pretraining seed, and the immune-task evidence doesn't fully cohere. The three downstream seeds used throughout quantify only fine-tuning variance, not variance from the MD-training/VQ-codebook stochasticity that produced the specific frozen encoder — a different MD seed (especially for VQ-VAE's stochastic codebook) could plausibly give a different encoder, and there's no way to know if the +0.174 interaction would replicate. On top of that, the cross-benchmark picture is genuinely mixed: VQ-VAE(D&D) loses on TABR-BERT S1–S4 (0.707→0.668), has a huge-variance borderline gain on AbRank (0.122±0.089 vs. a near-zero Base-ESM Spearman of −0.005±0.002), and its biggest nominal win (ITRAP, +0.454) is explicitly flagged by the authors as likely an artifact of a sub-random, high-variance baseline plus component recurrence.

Questions for authors
  1. Is Base ESM-2 35M in Tables 3–13 strictly the off-the-shelf checkpoint, or does it also get continued training (plain MLM, no MD head) on the same corpus for a matched number of steps? If not, can you report this no-MD control at least for the ProteinGym result — it directly decides whether the Section 4.2 interaction is about MD content or just in-domain continued training.
  2. How sensitive are the ProteinGym interaction (+0.174) and the BATCAVE/AbRank gains to the single MD-pretraining seed and VQ-VAE codebook stochasticity? Could you report at least one more independently-trained VQ-VAE codebook and/or MD-supervision seed for the D&D arm?
  3. For ITRAP (Table 10), Base-ESM's ROC-AUC of 0.374±0.062 is below chance (0.5) for a well-posed binary task — why? (label definition, class-weighting interaction with weighted BCE, optimization instability?) This bears directly on whether the +0.454 VQ gain is real or an artifact of an unstable baseline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.