acceptodds
Under review as a conference paper at ICLR 2027

Extracting and Steering Ensemble and Structural Properties in BioEmu

Abstract

Generative protein models have recently demonstrated strong performance in protein structure prediction tasks. In particular, the AlphaFold models are the standard for static structure prediction. Other models like BioEmu add additional functionality by generating protein conformations trained to sample thermally distributed conformations. However, we still lack a mechanistic understanding of how these models sample the distribution of structures. To improve our underlying understanding of these models, we train sparse autoencoders on BioEmu to identify interpretable features corresponding to structural and thermodynamic properties. In addition, we assess at what point in the model these properties are most highly represented. From this, we determine that the hidden activations can be used to predict desired ensemble properties from a single sampling trajectory. Further, we show that the hidden activations can be directly tuned to steer generation towards a desired property, as well as towards specific conformational domains like cryptic pocket opening, domain motion conformers, and kinase states. These results improve mechanistic understanding of protein models and open the door for targeted property and conformer steering of protein structure sampling models like BioEmu.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
BioEmu, interpretability, steering, SAE

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_qrwrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents three different contributions. 1) They train SAE on BioEmu representations and show that biophysical, thermodynamic and structural ensemble properties can be predicted from those. In particular they claim to predict DG and DDG values without sampling (as opposed to BioEmu. 2) They train a representation adapter to steer BioEmu towards specific radiuses of gyration. Then, they apply it as a steering force at a well-chosen diffusion timestep and show it effectively steers. 3) They train other small MLP adapters with an RMSD minimization objective to steer the generation towards specific protein conformations, which they evaluate on benchmark datasets with good results.

Using BioEmu is the main connection between the three parts of the paper (at least between 1 and 2-3). Line 139, "In this work, we steer with an SAE feature which correlates strongly with the protein radius of gyration": this sentence is misleading at the very least, since the steering and MLP adapters don't rely on the SAE features. From the code provided, I suspect it might have been the initial intention to use the SAE features, but it did not work.

Critical strengths

S1. The steering towards a given state ("open the potential cryptic pockets of this protein") is an interesting idea

Critical weaknesses

W1. Overall, the paper is underspecified and the claims are not precised. Reviewers (and readers) should not be expected to read the Appendix to understand the method presented. A few examples:

  • Line 139: "In this work, we steer with an SAE feature which correlates strongly with the protein radius of gyration": that’s not what is described in the paper
  • Line 160: " To this end, we correlate many different biophysical properties with feature activations in our trained SAEs." No description of the SAE training in the main, no pointer to the relevant appendix section.
  • Line 302: "To ensure the train and test splits were sufficiently different, we performed clustering on our full dataset and made the train/validation/test split based on those clusters." The datasets are only mentioned in the related work section, and it is unclear how the splitting is made in relation to the benchmark used.
  • Line 376: "In (Fig. 3d), the residual fraction at all timesteps for all cases is above 0.5". That's not what the figure shows. The domain motion exception is correctly mentioned but not the kinases, which remain below 0.5 for all timesteps.

W2. I don't think part 1 is novel, nor particularly interesting. I am not surprised to learn that the latent space of BioEmu incorporates those features. Your reported figures for DG prediction (around .56 spearman) do not match the ones reported in BioEmu (.67, fig 4a).

W3. For part 2, steering has only been studied on Rg (as stated in the limitations). It has not been compared to the many established steering methods, so the benefit of the strategy remains to be proven. Rg is differentiable wrt to coordinates, so existing steering methods are applicable.

W4. Part 3 relies on prior knowledge of the objective at hands (e.g. sampling DFG-in), contrary to the baselines. On domain motions, this results on an arguably arbitrary objective. On DFG-in/out, this corresponds to existing established collective variables that are not compared. I think the opening of cryptic pocket represent your most meaningful contribution. Despite this, the proposed approach does not result on sota performance.

W5. The experimental setup of part3 is not clean enough. As mentioned above, the splitting strategy wrt to the benchmark is not clear enough to me. Moreover, the results on cryptic pockets and domain motions are not really comparable to ConforNets, since the systems used are not the same. Considering the small number of systems per dataset, even a single system can have a non negligible impact. I would suggest presenting the results on the exact same systems as ConforNets for a fair comparison (or to run ConforNets on the same systems as the authors). As a side note, the results are presented with too many digits compared to your stds.

Questions for authors

Q1 In the Rg steering part, did you try updating the embeddings at multiple diffusion timesteps instead of a single one, similarly to the conformational sampling part ? Steering too early and too late is detrimental to the structure's validity as you showed, but applying steering at multiple intermediate steps could potentially refine the guidance towards the target property.

4 comments

We thank the reviewer for their helpful feedback. Our response will be split up into multiple comments due to the character limit. Based on their comments, we have substantially revised our claims, fixed incorrect lines, added supporting tests, and strengthened steering results. We hope our responses below address their concerns:

(1.1) Line 139: this is a mistake from a previous version, which has now been corrected. In the main text we only steer with the learned MLP. We also included the attempt to steer with SAE features in the appendix (Fig. A1), but it is less central to the main claims of the paper.

(1.2) Line 160: we have moved all of the work relating to SAEs to the appendix, as they are not central to the main claims of the paper. SAE details are in appendix sections (A.1-A.4). Our main contributions are extracting ensemble and free energy in a single pass, showing BioEmu’s activations can be directly manipulated to steer R_g at a single timestep, and showing that this is generalizable and can be extended to targeting specific conformation generation without a target structure or collective variable at inference time.

(1.3) Line 302: The datasets are listed in the Methods section, 3.2, lines 143 and 144 in the original submission. However, you are correct that we did not clarify how we split with respect to the benchmark. We have added this to the methods section, and clarified that we removed proteins which have over 30% sequence identity and over 50% coverage, which is the same sequence identity metric we used for making the train/validation/test split.

(1.4) line 376: this is a typo from before adding the kinase. The timestep analysis here is not central to our claim, and has been removed from the main text to improve focus.

(2) Thank you for the comment. Part 1 helps motivate steering the hidden representations directly, and we show that ensemble and free energies can be predicted with a single pass. Our results are calculated from the most updated bioemu-benchmarks github (https://github.com/microsoft/bioemu-benchmarks), which is closer to the Science publication of BioEmu [1], in which they say “BioEmu achieved a mean absolute error of less than 0.9 kcal/mol and a Spearman correlation coefficient of approximately 0.6 for the MEGAscale test proteins,” which is fairly close to our predictions. I believe the (0.67, Fig. 4a) you cite is from the older bioArxiv submission from 2024. To clear up any confusion, we recalculate the Delta G correlation from the bioemu-benchmarks github pipeline in (Fig. 1e).

(3) This is a good comparison to add, so we’ve added a comparison to BioEmu’s own Feynman-Kac (FK) steering method [1] with R_g as the collective variable in (Fig. 2f). We find that our steering outperforms BioEmu’s FK steering in the median for reaching the requested target (0.20 vs. 0.24, where perfect is 0), and also has fewer large errors.

(4) We originally chose the benchmark and success@B metric because it is used in the original BioEmu and ConforMix papers, but not only is it small, it also doesn’t capture what we’re truly trying to test (if you sample the targeted conformer more than once for a given sequence, the score remains the same). As such, we’ve moved the success@B tables to the appendix and switched to a separate pure percentage metric (originally presented in Fig. 3f, but not well highlighted), which scores the hit percentage of samples that are within the tolerance range. This tests targeted steering rather than broadening the sampling distribution, and is what our steering method is intended to do. The success@B metric measures whether at least one sample hits the target, but doesn’t distinguish between methods that hit the target multiple times, which doesn’t test targeted steering very well. Our relative performance on this metric is much clearer. The updated table is (Table 1, shown below), in which we match or outperform BioEmu on all targets, and perform better than ConforMix on most targets as well. We exclude ConforNets domain motions and cryptic pockets here, as you mention that they aren’t comparable due to ConforNets requiring per-protein training, which we agree with. We kept in the ConforNets kinase results as those aren’t trained per protein. The full ConforNets results are still present in the success@B appendix results. Most of the values on the BioEmu benchmark are within error due to the low number of systems, so we’ve also added our own larger test set to corroborate (Table 1, shown below), which shows our steering outperforming BioEmu and ConforMix. On domain motions, we agree that this is not an ideal metric, but it does provide some distinction, with the rest inferred from the input representations. For DFG-in/out, we compared to BioEmu’s FK steering [1] with the Kincore [2] DFG dihedral distance as the collective variable, in which we match on DFG-in and outperform on DFG-out (Table 2). We agree that the cryptic pocket result is the most consequential result, as it allows for steering without a target or collective variable at inference time, and does better than existing methods on our larger test set.

Benchmark panels

BenchmarkNBioEmuConforMixConforNetsOurs
CP (apo)320.08 ± 0.030.14 ± 0.05–0.11 ± 0.04
CP (holo)320.15 ± 0.050.20 ± 0.05–0.26 ± 0.06
DM (nearer)200.45 ± 0.080.16 ± 0.03–0.59 ± 0.09
DM (further)200.12 ± 0.040.16 ± 0.03–0.16 ± 0.06
Kin. (DFG-in)200.55 ± 0.090.42 ± 0.080.65 ± 0.090.55 ± 0.09
Kin. (DFG-out)200.22 ± 0.060.30 ± 0.080.37 ± 0.090.30 ± 0.08

Held-out test sets

BenchmarkNBioEmuConforMixOurs
CP (apo)1340.11 ± 0.020.087 ± 0.0160.17 ± 0.03
CP (holo)1340.078 ± 0.0170.12 ± 0.020.16 ± 0.03
DM (nearer)690.34 ± 0.050.114 ± 0.0160.42 ± 0.05
DM (further)690.09 ± 0.020.038 ± 0.0080.14 ± 0.04

(5) We addressed the splitting strategy above in (1.3). We agree that the results are not comparable to ConforNets as ConforNets is trained on each individual protein, and is also not tested on the exact same set. As such, we’ve removed ConforNets from our evaluated metric for cryptic pockets and domain motions, mentioned in (4), but have the full success@B with consistent systems in the appendix (Table A2-A5) and retain the kinases since those aren’t per protein. We’ve also fixed the digits.

Q1: We originally did not try updating the embeddings at multiple diffusion timesteps. We’ve now added this to (Fig. 2b). We find that performance is very close to the best single site, which makes sense as none of the other timesteps had much of an effect.

[1] Lewis et al. Scalable emulation of protein equilibrium ensembles with generative deep learning, 2025. [2] Modi et al. Kincore: a web resource for structural classification of protein kinases and their inhibitors, 2021.

Reviewer_qrwrOpenReview ↗

thanks for your anwser

Thanks for your answer.

1/2. Thanks for fixing the specifics, but I still think the paper should focus on the steering method and its validation.

  1. Thanks for comparing to FK, but you could also include conformix, conforflux, confornets, etc..

4/5. You state "We exclude ConforNets domain motions and cryptic pockets here, as you mention that they aren’t comparable due to ConforNets requiring per-protein training, which we agree with." But my statement was "not really comparable to ConforNets, since the systems used are not the same." since you had 30 and not 32 systems.

You should include confornets results for all datasets, since they seem to outperform your method. To make the computational aspects clear, you can include a column with the overall time. A possibility would be to draw metric curves with time and num_samples as the x-axis.

Overall, this does not really address my concerns that:

  1. the first parts about R_g and representation analysis are not interesting enough for ICLR. I think a deeper rewrite is needed that directly investigate the steering aspects of the project
  2. the part about steering is unsufficiently benchmarked and compared to SOTA, especially considering its limited applicability (DFG-in/out) and intuition (for the domain motion).

I choose to maintain my rating

Thank you for your quick response.

(1/2) We have now expanded the steering analysis, including a distance-stratified analysis and compute cost comparison.

(3) ConforMix, ConforFlux, and ConforNets are all diversification methods. In their diversification configurations, there isn’t a way to specify a level (e.g. sample R_g one standard deviation from the mean), so they would each sample the same mean R_g regardless of what R_g we request from steering. Our method, as well as BioEmu’s FK steering, targets a specific R_g, which is why we chose to compare to FK and why it doesn’t make much sense to compare to diversification methods for this.

(4/5) Thank you for the comment. We have now added back ConforNets to the BioEmu benchmark, evaluated on the same systems as all of the other methods (we dropped one domain motion system that didn’t fit in H200 RAM for ConforNets). We are unable to run ConforNets on our own test set, as it would take too long to run within the remaining rebuttal period (1000+ GPU-hours). In fact, this is one of the main drawbacks of ConforNets compared to our method. At your suggestion, we have added a plot (Fig. 3b) showing the compute difference between the different benchmarked methods, with ConforNets coming out 10-20x more expensive in GPU-hours for cryptic pockets and domain motions than our method, ConforMix, and BioEmu. This is potentially very important for applications like high throughput screening, where our method would be an order of magnitude faster. We have also now emphasized in the main text that our method works comparatively better on systems with targets close to the unsteered ensemble, whereas diversification methods tend to work comparatively better on systems with targets far from the unsteered ensemble that BioEmu very rarely samples, which is a more specific claim. In (Table 1, below), we show that at shifts within 0-2 units of the success threshold, we outperform BioEmu and ConforMix (matching at 1-2 for kinases), and either match or outperform ConforNets at a tenth of the compute cost, with the exception of kinases at 1-2. Meanwhile, at large shifts, we perform competitively with BioEmu and ConforMix, and worse than ConforNets. As such, for systems that target structures close to the unsteered ensemble, our method is better than the other benchmarked methods either in compute cost or in pure performance, with the kinase exception.

BioEmu benchmark set

Benchmarkd₀/τNBioEmuConforMixConforNetsOurs
CP< 140.78 ± 0.070.85 ± 0.090.981 ± 0.0190.985 ± 0.015
1–2130.27 ± 0.070.36 ± 0.100.45 ± 0.090.43 ± 0.09
≥ 2470.011 ± 0.0040.059 ± 0.0190.14 ± 0.030.050 ± 0.016
DM< 170.75 ± 0.080.17 ± 0.030.58 ± 0.060.951 ± 0.016
1–2140.30 ± 0.060.20 ± 0.030.40 ± 0.060.44 ± 0.10
≥ 2170.10 ± 0.050.12 ± 0.040.25 ± 0.060.14 ± 0.07
Kin.< 1140.83 ± 0.030.63 ± 0.070.87 ± 0.070.86 ± 0.02
1–2180.20 ± 0.050.31 ± 0.080.42 ± 0.100.27 ± 0.06
≥ 280.00 ± 00.00 ± 00.08 ± 0.080.00 ± 0

Held-out test sets

Benchmarkd₀/τNBioEmuConforMixOurs
CP< 1160.78 ± 0.040.41 ± 0.080.96 ± 0.02
1–2640.14 ± 0.030.23 ± 0.040.33 ± 0.05
≥ 21880.018 ± 0.0060.032 ± 0.0060.040 ± 0.012
DM< 1240.78 ± 0.050.23 ± 0.020.93 ± 0.04
1–2380.23 ± 0.040.106 ± 0.0170.37 ± 0.06
≥ 2760.020 ± 0.0090.013 ± 0.0050.029 ± 0.013
Reviewer_yMwrOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies BioEmu, a pretrained diffusion model that generates protein conformations from a protein sequence to approximate its equilibrium ensemble. The method is to use an sparse autoencoder and linear regression, and train a lightweight MLP, called adapter, to control radius of gyration and conformational states.

Critical strengths

The paper aim to discover the potential to steering a pretrained, forzen generative model with a MLP. The paper have found some interesting empirical results from applying the steering at different time step of the diffusion model.

Critical weaknesses

The paper does not draw out a clear interpretation of the steering method. The MLP adapters modify the raw activations without using the SAE feature. The success provides limited evidence that the feature extracted by SAE can control the desired properties causally. Additionally, in this paper, time-aware SAEs follow TIDE. FoldSAE already uses SAEs to interpret and steer RFdiffusion [1], while earlier ESMFold work demonstrates structural steering at fixed sequence [2]. Both works are missing from the reference of the paper. The contribution of this paper should be limited around BioEmu ensemble readout and broader transfer. References: [1] Zarzecki et al., FoldSAE, 2025. [2] Parsan et al., Towards Interpretable Protein Structure Prediction with Sparse Autoencoders, 2025.

Questions for authors

Does direct intervention on selected SAE features reproduce the steering effects? How does performance vary with similarity to training proteins?

3 comments

We thank the reviewer for their helpful feedback. We have substantially revised our claims, added new baselines, and strengthened steering results. We hope our responses below address their concerns:

(1) We have removed the mechanistic and interpretability claims and moved the SAE work to the appendix, as it is not central to the main contributions of the paper, which are predicting ensemble properties from a single generation, property steering, and conformation steering without a target structure or collective variable at inference time. The steering method is indeed the learned MLP adapters, motivated by the representation analysis on the full hidden activations. We agree that FoldSAE [1] and the work by Parsan et al. [2] are relevant, and both papers have been added in the related works section. However, our primary contribution differs from these works in that we use learned MLP adapters rather than SAE features to steer, and we target generation of specific conformers on unseen proteins without requiring the target at inference time.

Q1: Direct intervention on selected SAEs does not reproduce the MLP steering effects. These results are presented in (Fig. A1c), in which SAE features are not able to steer R_g. Almost none of the R_g steering training proteins exceed the sequence similarity threshold with the test proteins under our clustering metric of greater than 30% identity and 50% coverage, so sequence similarity across all the proteins is low.

[1] Zarzecki et al., FoldSAE, 2025. [2] Parsan et al., Towards Interpretable Protein Structure Prediction with Sparse Autoencoders, 2025.

In response to additional reviewer feedback, we have expanded our conformation steering benchmark (Table 1, below) and added compute analysis (Fig. 3b). We wanted to highlight these additional results as they are relevant to your concerns as well.

We have added back ConforNets to the BioEmu benchmark, evaluated on the same systems as all of the other methods (we dropped one domain motion system that didn’t fit in H200 RAM for ConforNets). We are unable to run ConforNets on our own test set, as it would take too long to run within the remaining rebuttal period (1000+ GPU-hours). In fact, this is one of the main drawbacks of ConforNets compared to our method. We have added a plot (Fig. 3b) showing the compute difference between the different benchmarked methods, with ConforNets coming out 10-20x more expensive in GPU-hours for cryptic pockets and domain motions than our method, ConforMix, and BioEmu. This is potentially very important for applications like high throughput screening, where our method would be an order of magnitude faster. We have also now emphasized in the main text that our method works comparatively better on systems with targets close to the unsteered ensemble, whereas diversification methods tend to work comparatively better on systems with targets far from the unsteered ensemble that BioEmu very rarely samples, which is a more specific claim. In (Table 1, below), we show that at shifts within 0-2 units of the success threshold, we outperform BioEmu and ConforMix (matching at 1-2 for kinases), and either match or outperform ConforNets at a tenth of the compute cost, with the exception of kinases at 1-2. Meanwhile, at large shifts, we perform competitively with BioEmu and ConforMix, and worse than ConforNets. As such, for systems that target structures close to the unsteered ensemble, our method is better than the other benchmarked methods either in compute cost or in pure performance, with the kinase exception.

BioEmu benchmark set

Benchmarkd₀/τNBioEmuConforMixConforNetsOurs
CP< 140.78 ± 0.070.85 ± 0.090.981 ± 0.0190.985 ± 0.015
1–2130.27 ± 0.070.36 ± 0.100.45 ± 0.090.43 ± 0.09
≥ 2470.011 ± 0.0040.059 ± 0.0190.14 ± 0.030.050 ± 0.016
DM< 170.75 ± 0.080.17 ± 0.030.58 ± 0.060.951 ± 0.016
1–2140.30 ± 0.060.20 ± 0.030.40 ± 0.060.44 ± 0.10
≥ 2170.10 ± 0.050.12 ± 0.040.25 ± 0.060.14 ± 0.07
Kin.< 1140.83 ± 0.030.63 ± 0.070.87 ± 0.070.86 ± 0.02
1–2180.20 ± 0.050.31 ± 0.080.42 ± 0.100.27 ± 0.06
≥ 280.00 ± 00.00 ± 00.08 ± 0.080.00 ± 0

Held-out test sets

Benchmarkd₀/τNBioEmuConforMixOurs
CP< 1160.78 ± 0.040.41 ± 0.080.96 ± 0.02
1–2640.14 ± 0.030.23 ± 0.040.33 ± 0.05
≥ 21880.018 ± 0.0060.032 ± 0.0060.040 ± 0.012
DM< 1240.78 ± 0.050.23 ± 0.020.93 ± 0.04
1–2380.23 ± 0.040.106 ± 0.0170.37 ± 0.06
≥ 2760.020 ± 0.0090.013 ± 0.0050.029 ± 0.013

Thank you for the rebuttal. I maintain my original assessment.

Reviewer_o5pPOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

In this paper the authors investigate the learned representations of BioEmu, which is a generative protein model designed to approximate equilibrium conformational ensembles. In particular, the goal is generation with property control; that is, to determine whether manipulating BioEMU's hidden activations can steer the generation of protein structures towards desired properties and conformational states.

Sparse autoencoders (SAEs) are used/trained to identify features associated with structural and thermodynamic properties potentially encoded in the BioEMU's learned representations. The paper also evaluates property prediction across several denoising timesteps and trains MLP adapters to steer/control for radius of gyration (R_g), cryptic-pocket states, kinase conformations, and domain motions.

The work presents a potentially useful approach to steering in activation space and provides some evidence of generalization to proteins held out from adapter training. However, the evidence for mechanistic interpretability, thermodynamic insight, and broadly superior conformational control is substantially weaker than the paper’s own framing and conclusions. The most accurate conclusions are actually provided honestly by the authors in the limitations section.

Critical strengths

(1) Understanding and controlling the representations of generative protein models is both timely and important. Parallel work has happened in small molecule generation (with property control), though the authors do not connect with that work. "Targeted generation," as the authors frame it, is important, even though the authors seem to mainly focus on its potential to reduce the cost of obtaining specific conformations (in comparison to "classic/physics-based" MD- and/or MC-based approaches for the previous decades).

(2) The authors examine predictive features across denoising timesteps and demonstrate that supervised interventions can bias generated structures toward specified properties and conformational states. The multi-timestep steering results on held-out proteins are indeed an interesting/promising proof of concept.

(3) The inclusion of cryptic pockets, kinase states, and domain motions is useful in broadening the investigation beyond the control of single scalar properties, and the authors provide useful comparisons with existing relevant approaches.

Critical weaknesses

(1) The demonstrated/achieved correlation between SAE features and structural properties does not establish the mechanistic understanding that the authors (over)claim in the paper. The full raw activations are shown to generally outperform individual SAE features. Moreover, the steering adapter modifies hidden activations directly rather than intervenes on identified SAE features. Therefore, the touted causal connection between interpretable features and successful steering has not been established in the paper in any convincing manner.

(2) The predictability of the radius of gyration (R_g) should be compared with a sequence-length-only baseline. R_g is a trivial property; there is a classic formula that predicts it fairly well from just polymer theory (Flory's scaling law) from equilibrium structures. Moreover, the marked performance decline after within-sequence normalization deserves a more careful and honest interpretation than currently given in the paper. The results also appear to capture substantial between-protein variation without clearly establishing how much sample-specific information is actually represented. Likewise, the folding-stability results show only marginal improvement over Evoformer representations, which seems to suggest the authors should substantially limit their claims about the contributions of BioEmu's denoising computation.

(3) The main text in the paper demonstrates single-sample prediction of folding stability. On the other hand, the appendix describes averaging four samples per sequence. This discrepancy needs to be addressed, alongside a clear computational comparison with extended sampling.

(4) Structural-retention and geometric constraints are useful to control for in generation, but the paper does not sufficiently characterize off-target structural changes, conformational diversity, or the effect of steering on ensemble properties. I caution the authors that reaching a target structure more frequently is not equivalent to accurately characterizing the associated conformational distribution.

(5) Improvements over BioEmu and ConforMix are inconsistent across benchmarks. The test sets are also quite smal. I also find the comparisons are not uniformly controlled. As an exemplar here, the claimed cross-family generalization requires qualification, particularly because the kinase set was excluded from sequence-identity clustering. The conclusions are exaggerated and need to instaed reflect the narrower provided evidence for a useful steering method. As currently, framed, they imply a broadly established mechanism for interpretable, generalizable conformational control not supported by the evidence.

Questions for authors

The paper shows that supervision from known conformations can train an adapter to bias a pretrained generator toward similar conformations in held-out proteins. Has the adapter learned the physical mechanism underlying the conformational change?

If physics is beyond the scope, and if the target structure is already known for the protein in question, why is this methodology even needed to reach it? The relevant evidence would be the reliable generation of a valid, previously-unknown target conformation.

What scientific knowledge does targeted conformer steering yield beyond the ability to bias generation toward a supervised structural endpoint? If the aim is to understand conformational transitions, the paper must establish something about intermediate states or physical process. In this context, denoising steps cannot be interpreted as mechanistic stages. If the aim is just endpoint generation, the contribution should be assessed as a method for predicting previously-unknown conformations. This necessitates appropriate comparisons to simpler alternatives as well as evidence of structural validity.

6 comments

We thank the reviewer for their helpful feedback. Our response will be split up into multiple comments due to the character limit. Based on their comments, we have made substantial revisions to the manuscript claims, strengthened steering results, and added supporting tests. We hope our responses below address their concerns:

(1) This is a good point, and mechanistic and interpretable were the wrong words to use here. They appear in the abstract, and have since been reworded to express our actual claim: structural and ensemble properties can be extracted with high correlation from the full hidden activations, which motivate steering from the hidden activations. We retain the wording “representational understanding,” as we do show that the representations contain extractable properties, and can be transformed to steer along properties and conformations. We have also moved all of the work relating to SAEs to the appendix, as they imply an interpretability claim that is tangential to our main contributions.

(2.1) Your first point: a metric to measure the effects of trivial properties like sequence length was indeed missing, and we have added it in (Fig. 1d). From the updated figure, pure sequence length prediction fit with a power law actually performs quite poorly (R^2 around 0.1 for R_g), as within-sequence variance of R_g is quite significant. This is due to many chains being able to be more extended or compacted depending on the specific generation sample, requiring more information than just sequence length to predict. After subtracting off the pure sequence length prediction, R^2 values fall only marginally for most properties.

(2.2) Your second point: there is indeed a large performance decline after within-sequence normalization, which we failed to address originally. Importantly, the way we conducted the within-sequence normalization was through z-score normalization, which removes much more than just sequence length, including compactness, fold, etc. This is helpful for determining the predictive power within-sequence, but does remove much more than just sequence length. Upon closer analysis, a decent portion of the drop in R^2 comes from dividing by the standard deviation, as some proteins already have very small standard deviations, which causes even a close absolute prediction (and physically negligible gap to the actual R_g) to become very bad. As such, it’s a less suitable metric for our systems. We’ve replaced this with a metric in which we only subtract the mean, which still shows the sample vs. sample predictability, but without the large effect of the sequences with small standard deviations. The R^2 values for subtracting off the mean are lower than the original, which is expected as the between protein effects are discounted, but the gap is much smaller and still reasonably high.

(2.3) Your third point: we addressed this in the original submission on page 5, “The SAE latents and raw BioEmu activations performed only marginally better than the Evoformer priors, indicating that BioEmu denoising may be encoding important information for folding stability, but that much of the representations are already encoded in the Evoformer embeddings (Fig. 1c).” We agree that the Evoformer representations are fairly close to the BioEmu representations, and that much of this information is encoded in both the Evoformer and BioEmu representations, with the BioEmu representations adding only a marginal amount. This in itself is a result, as the denoising computation may not be adding much with regards to Delta G. However, it still stands that a single pass extraction from BioEmu’s (and the Evoformer’s) embeddings has comparable performance to extended sampling of BioEmu to obtain this property. We’ve updated the wording in this section slightly, but mainly convey the same message in the main text as is written here. It is now, “The raw BioEmu activations performed only marginally better than the Evoformer priors, indicating that BioEmu denoising may be encoding important information for folding stability, but that much of the representations are already encoded in the Evoformer embeddings (Fig. 1c).”

(3) Thank you for bringing this up. This is a slight wording mistake. Our intention was to say that only 1 sample is used for any correlation calculation, but that we conducted this over 4 seeds to reduce the variability of the mean that we report, as some seeds had higher correlations and some seeds had lower correlations. That is to say, we calculated the correlations for each seed separately, and then averaged the correlations after. For transparency, we have added the full table of seeds to (Table A1). To compare with extended sampling, we took the values from BioEmu’s bioemu-benchmark github (https://github.com/microsoft/bioemu-benchmarks) and added them in a plot next to our own predictions (Fig. 1e,f). The predictions for extended sampling come directly from the pipeline published on the bioemu-benchmarks github (which reproduce the results in the 2025 Science publication of the paper), and our own computation is directly from our ridge regression probe on BioEmu’s activations using out-of-fold predictions in 5-fold cross validation. For computational cost, BioEmu [1] states they use 200-10,000 generations per protein, depending on the protein, to obtain the Delta G and Delta Delta Gs, while we only require one. We’ve added this to the main text.

(4) To be clear, our goal is not to capture the full conformational distribution, but to target generation of a specific conformer (e.g. apo cryptic pocket). This may have been a point of confusion due to our comparing to ConforMix, which does preserve the conformational distribution. To better characterize conformational diversity, off-target structural changes, and ensemble properties, we have added (Table 4), which covers all 3 of these. Through a decreased pairwise RMSD, we find that we narrow, but don’t collapse the distribution, which is the intention of our adapter. Structural lDDT, as well as mean R_g, helix percentage, and strand percentage are preserved, indicating small off-target structural changes and small change in ensemble properties. However, we maintain that equilibrium ensemble properties should not be calculated from the steered structures.

(5.1) We originally chose the benchmark set and success@B metric because it is used in the original BioEmu and ConforMix papers, but not only is it small, it also doesn’t capture what we’re truly trying to test (if you sample the targeted conformer more than once for a given sequence, the score remains the same). As such, we’ve moved the success@B tables to the appendix and switched to a separate pure percentage metric (originally presented in Fig. 3f, but not well highlighted), which scores the pure percentage of samples that are within the tolerance range. This tests targeted steering rather than broadening the sampling distribution, and is what our steering method is intended to do. The success@B metric measures whether at least one sample hits the target, but doesn’t distinguish between methods that hit the target multiple times, which doesn’t test targeted steering very well. The updated table is (Table 1), in which we match or outperform BioEmu on all targets, and perform better than ConforMix on most targets as well. We exclude ConforNets cryptic pocket and domain motions here, as one of the other reviewers mentioned that they aren’t comparable due to ConforNets requiring per-protein training, which we agree with. We kept in the ConforNets kinase results as those aren’t trained per protein. The full ConforNets results are still present in the success@B appendix results. Most of the values on the BioEmu benchmark are within error due to the low number of systems, so we’ve added our own test sets to corroborate (Table 1, shown below), which are substantially larger, and show our models outperforming both BioEmu and ConforMix. Our weakest metric is kinases, which ConforNets outperforms us on, although this is still within error. However, our cryptic pocket and domain motion performance is better than both BioEmu and ConforMix on our test set, which has more systems than the original benchmark.

Benchmark panels

BenchmarkNBioEmuConforMixConforNetsOurs
CP (apo)320.08 ± 0.030.14 ± 0.05–0.11 ± 0.04
CP (holo)320.15 ± 0.050.20 ± 0.05–0.26 ± 0.06
DM (nearer)200.45 ± 0.080.16 ± 0.03–0.59 ± 0.09
DM (further)200.12 ± 0.040.16 ± 0.03–0.16 ± 0.06
Kin. (DFG-in)200.55 ± 0.090.42 ± 0.080.65 ± 0.090.55 ± 0.09
Kin. (DFG-out)200.22 ± 0.060.30 ± 0.080.37 ± 0.090.30 ± 0.08

Held-out test sets

BenchmarkNBioEmuConforMixOurs
CP (apo)1340.11 ± 0.020.087 ± 0.0160.17 ± 0.03
CP (holo)1340.078 ± 0.0170.12 ± 0.020.16 ± 0.03
DM (nearer)690.34 ± 0.050.114 ± 0.0160.42 ± 0.05
DM (further)690.09 ± 0.020.038 ± 0.0080.14 ± 0.04

(5.2) The generalization applies only to the cryptic pockets and domain motions, as we did not conduct clustering on the kinase set. The generalization claim that results from this comes from the cryptic pocket and domain motion results, not the kinase results. We have also adjusted our claim of mechanistic understanding and interpretability, as talked about in point 1. However, we still maintain that the results do generalize, as all of our reports are on unseen proteins with no target at inference time, and the train and test sets were clustered before the split.

Q1+Q2: Questions 1 and 2 are related, so we’ll answer them together. First, the adapter has not learned the physical mechanism in the sense that you would not be able to extract a transition state from one state to another, and understanding the transition state was not the goal of the adapter. It does, however, show that there exists some general transformation in the hidden representations to target a specific conformer for unseen proteins. Second, the target structure does not have to be known, as we do not use it at inference time other than to check if our generated structure matches after generation is finished. We run on sequences where the target structure is known because we are either training (which requires the label) or testing (which also requires the label to know if our generation is correct). However, this does not mean that it does not work on structures without the known target structure. In fact, that is the whole point of separating training and test data. By separating the two groups, we show that the adapter generalizes to unseen proteins. The closest we can get to “reliable generation of a valid, previously-unknown target conformation” is with our unseen test set, as the only way we can fully validate a generation is by knowing the target in the first place, which would invalidate the sequence as a previously-unknown target conformation.

Q3: The aim is not to understand conformational transitions, but to instead establish that it is possible to learn a generalizable transformation in BioEmu’s representations that can steer towards a specified conformation. The contribution is that we can steer towards these specific conformations without a target at inference time. We are not aware of any other steering methods that do this without a target structure or specified collective variable at inference time. The closest ones are the ones that spread conformation distribution sampling (like ConforMix, which we compare to), or ones that require the target structure or a defined collective variable at inference time, but these are often not definable for unknown structures, like with cryptic pockets and domain motions. For structural validity, we have added (Table 3), which compares our steered structures to ConforMix and BioEmu unsteered structures in Ramachandran outliers and bond failures. Our structures are slightly worse than BioEmu’s unsteered structures, but about the same as ConforMix’s structures, making our steering method comparable in validity to other steering methods.

[1] Lewis et al. Scalable emulation of protein equilibrium ensembles with generative deep learning, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.