- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper connects a frozen protein ensemble generator, BioEmu, and a frozen cryo-EM image encoder, Cryo-IEF, through a learned low-dimensional shared representation. It constructs an uncertainty-weighted likelihood for guiding conformational sampling and uses simulated candidate-image distributions to estimate conformational populations. The approach is evaluated using simulated particles generated from BioEmu conformations, PDB structures, and an independent molecular-dynamics ensemble. The experiments demonstrate shifts toward target conformations and recovery of imposed population landscapes under the tested simulation conditions.
Critical strengths
-
The shared representation provides an interesting interface between pretrained structural and image models. Using the same code for both conditional generation and population estimation gives the approach a coherent methodological foundation and allows the pretrained models to remain frozen.
-
The statistical modeling is carefully developed. The guidance accounts separately for image variability and time-dependent uncertainty in the generator representation, while population estimation explicitly models imaging-induced scatter. These design choices address substantive inference issues and constitute the main technical strength of the work.
Critical weaknesses
-
The guidance task has an unresolved dependency on the conformational identification it is intended to support. Section 3.3 requires particles from a single conformation, which the synthetic experiments obtain directly from known structure identities. The discussion proposes using 3D classification or heterogeneous reconstruction to supply these particles. However, if existing methods cannot reliably identify and isolate a rare state, they cannot provide the required input; if they can, three-dimensional structural information is already available, and the proposed method must demonstrate an additional benefit over reconstruction, refinement, or structural modeling using that information. The paper does not establish this intermediate regime. Furthermore, a conformational class in continuously heterogeneous data is not equivalent to the experimental setup of 128 images generated from exactly one structure. Selecting a narrow conformational neighborhood introduces a trade-off between homogeneity, particle count, and viewing coverage, but the required homogeneity and tolerance to residual variation are not evaluated. These issues leave the practical applicability of the guidance task unresolved, even before considering simulation-to-experiment transfer.
-
The population estimates lack a demonstrated basis for interpretation as reliable scientific measurements. All particle-image experiments are simulated, including the PDB and MD evaluations. Recovering imposed populations under the chosen simulator does not establish that fitted weights represent physical populations in experimental data. Missing candidate structures, inaccurate imaging distributions, and biases in the learned representations can all potentially be absorbed into those weights. A good fit in the shared feature space therefore does not independently distinguish correct populations from compensation for model mismatch. The paper does not establish how this distinction would be made or how incorrect estimates would be detected. The independent MD experiment illustrates this concern: Appendix I.5 reports that the displayed landscape uses a code-offset correction measured using the ensemble’s known structural coordinate, reducing the error from 0.57 to 0.10 . The availability of equivalent information in the intended application is unclear. My concern is therefore that the current evaluation does not establish a credible route from successful synthetic recovery to trustworthy inference on experimental data.
Questions for authors
-
Practical workflow and added value: Beyond the proposed use of upstream 3D classification, what concrete experimental setting allows a rare conformational class to be reliably isolated while leaving a structural problem that the proposed guidance demonstrably improves? What structural information would already be available, and how would the method compare with reconstruction, refinement, or candidate selection using comparable information and computation?
-
Validity of the input assumption: What degree of within-class conformational variation can the guidance tolerate? Is there evidence that particles selected from a realistic conformational neighborhood, rather than rendered from one fixed structure, provide a sufficiently accurate target code without introducing misleading averaging effects?
-
Population validation: What independent validation or diagnostic would distinguish genuine population changes from compensation for missing candidate states, imaging mismatch, or representation bias? Please clarify what evidence currently supports this distinction and what remains a proposed future validation strategy.
-
MD calibration: Appendix I.5 describes a code-offset correction measured using the MD ensemble’s known structural coordinate. How could this correction be estimated when the target conformations are unknown, and how should the displayed MD result be interpreted if that information is unavailable?
7 comments
I appreciate the authors' efforts to address my concerns, particularly by including experiments on real data. However, my major concerns remain: the existing experiments do not demonstrate a complete workflow on real data that would establish the method's applicability to real cryo-EM tasks. The experiments on real data still bypass the classification issue, directly using well-analyzed classification results from published datasets, which were obtained using conventional pose-search-based reconstruction methods. More importantly, the limitations and potential biases associated with pose estimation and density reconstruction are inherited by the downstream guidance pipeline through these pre-classified particles. In other words, pose estimation and density reconstruction are implicitly performed during upstream classification, which undermines the claim that the method can provide guidance without maps or poses.
Moreover, the results on real data are still unsatisfactory. Substantial mismatches with the deposited atomic models can be observed in the qualitative comparisons. Quantitative results also show relatively large differences, especially for PaBphP. Besides, the population results further indicate that the bias introduced by the proposed conformational population inference cannot be ignored.
At this stage, I would consider the proposed method to be an interesting idea, yet it appears unsuitable for practical cryo-EM workflows. I would like to see the authors demonstrate how the proposed method can be applied to real data, for example, by conducting the same workflow used in Appendix E.2 on TmrAB and PaBphP.
Thank you for the concrete request. In the revised App. J we guide both proteins by unsorted particles, without the classes of the depositors (Table S19). The guided samples also lie closer to the deposited models, now that we model radiation damage in the simulated images and correct the offset between the embeddings of experimental particles and simulated particles (Table S18, Figure S1).
Guidance without classes. We take the deposited particles without their class labels and fit the population model of §3.4 to them. We then guide each sample toward a conformation drawn from the fitted population. We do not guide by the unsorted particles directly, because a set that mixes two states shifts the samples toward the majority state (App. E.2).
For phytochrome, we guided 24 samples toward an open conformation, and all became open, while no unguided sample became open. These 24 samples lie at a Cα RMSD of 0.54 nm from the deposited open model, and the unguided samples that start from the same noise lie at 0.79 nm. For the transporter, 35 samples were guided toward an inward-facing conformation, and 31 became inward-facing, compared with 2 of the unguided samples that started from the same noise.
Bias inherited from classification. In these runs we assign no particle to a class, estimate no orientation and build no map. However, the runs still rely on the depositors for picking, cleaning, and centring the particles, and for the defocus values, and App. J lists this as a limit.
Agreement with the deposited models. The agreement is closer in the revision. Samples that are guided by the open class of the phytochrome lie 0.35–0.48 nm from the deposited open model, and unguided samples lie 0.8 nm from it. Samples that are guided by the inward-facing class of the transporter lie 0.24–0.26 nm from the deposited inward-facing model, and unguided samples lie 0.7 nm from it. Figure S1 shows the guided samples next to the deposited models.
Populations. We agree that the absolute shares are biased. In the runs without classes, the fitted share of the smaller state is 16% for the phytochrome and 25% for the transporter, and the depositors assigned 36% of the particles to it. The method recovers changes in share better than the shares themselves. On mixtures of known composition, the estimated share rises with the true share at a slope of about 0.8 on both proteins. We therefore limit our claims about populations to changes between the particle sets of one dataset.
Thank you for raising two points on which the method's usefulness depends: where its input comes from and whether its population estimates can be trusted.
W1, Q1. What guidance adds to classification.
The dilemma is well put, and we tested both of its cases.
When classification works. The transporter's first-round classes (8.6 Å resolution) come from the depositors' pose-based processing, so a low-resolution map of each class exists. From about 1,000 particles of one such class, guidance produces atomic models within 0.25–0.43 nm of the deposited inward-facing models (unguided: 0.65–0.71 nm).
When classification cannot isolate the state. We simulated 4,000 particles of the 57 kDa GroEL subunit at the paper's signal-to-noise ratio. They come from two equally populated conformational clusters, 1.1 nm apart along the opening coordinate. Even given the true poses, RELION and cryoDRGN separate the two clusters with only 52–61% purity (chance: 50%). Our population model assigns the same particles with 93% purity and 72% recall, and guidance weighted by its assignments is within 0.03 nm of guidance on perfectly sorted particles. It can do so because it compares the particles with the generator's candidate structures, which RELION and cryoDRGN do not have.
Comparison with reconstruction. In simulation with the true poses, a map from 32 particles identifies the closest candidate structure more accurately than guidance from 128. We therefore see two uses for guidance. One is the first case above, where guidance provides an atomic model for each coarse class without high-resolution refinement. The other is for particles whose poses cannot be determined, as in the subunit above, where RELION's refinement did not recover the poses (median error 88–124°; random: 132°).
W1, Q2. Variation within a class.
In simulation, the classes do not have to be narrow. A class of 128 different conformations that follow the population density gives a median core RMSD within 0.03 nm of 128 images of its central member. Guidance improves on unguided sampling in all twelve classes, with members within a median of 0.36 nm of the central member, compared with 0.41 nm for the widest class we tested.
The target code is the mean over the class. A class that mixes two distinct states (1 nm apart, 90 and 38 particles) therefore gives samples pulled toward the majority state, as a consensus map would be, and such data should be split first.
Four to eight particles of a narrow class already reduce the distance to the extended GroEL target from 1.82 to 0.56–0.70 nm. Viewing coverage has a larger effect, because top views alone give 1.53 nm and side views alone 0.69 nm (all views: 0.53 nm).
W2, Q3. Populations, or compensation for mismatch?
On experimental particles, the estimates follow changes in composition. As the reviewer says, a good fit alone cannot show this.
Current evidence. We therefore mixed experimental particles of two deposited classes at known ratios from 0 to 100% and estimated the share without the labels. All three kinds of mismatch the reviewer lists are present but identical in every mixture; only the composition changes. On both proteins, the estimated share rises at every step, with a slope of about 0.8 on the transporter and about 0.3 on the phytochrome (1 would be ideal). Mismatch thus compresses the scale and adds an offset, but the direction of change is recovered. We therefore claim only changes in population between datasets processed in the same way.
Future validation. We still need datasets whose state fraction is set biochemically and measured independently, such as a ligand titration.
W2, Q4. The MD calibration.
It can now be estimated from the particles and the generator's samples, without the MD structures. We fit a single offset of the particle embeddings jointly with the populations. One component of this offset changes the embeddings in the same way as moving the population along the coordinate, and the joint fit cannot separate the two. We set this component with a linear map from the embedding to the coordinate, learned on the generator's samples. The landscape error is then 0.12 kBT RMS, compared with 0.57 kBT uncorrected and 0.10 kBT with the true structures. The test covers one MD ensemble, so for experimental data we still rely on the mixture test above.
The changes are collected in the general response. If the new experiments settle these two points, we would ask you to consider updating your assessment. Where they fall short, we would be glad to discuss the point further.