acceptodds
Withdrawn from ICLR 2027

Learning the Shared Latent Structure of Cryo-EM Inverse Problems

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Understanding biomolecular function requires characterizing three-dimensional structure and conformational variability, for which cryogenic electron microscopy (cryo-EM) is a powerful tool. Cryo-EM produces noisy two-dimensional particle images in which molecular structure, particle pose, and imaging conditions are jointly encoded. Recovering biological information from these observations spans inverse problems including pose estimation, heterogeneity analysis, and reconstruction. These tasks are usually addressed independently despite shared protein structure and image formation, motivating a common representation across inverse problems. Since biologically meaningful differences may be spatially localized, we study self-distillation with complementary global and masked patch objectives. Because experimental dataset discrimination can reflect acquisition-specific cues, we develop controlled synthetic and experimental evaluations and find that structural discrimination benefits from dense spatial features. We introduce the _EMber_ family of vision transformers, pre-trained on >50 million experimental particle images from 214 EMPIAR datasets together with simulated projections from 268,000 molecular assemblies. _EMber_ improves upon existing methods at conformational and compositional discrimination by 14.85 and 14.26 percentage points, estimates conformational populations to within 1.59pp error with a linear probe, and produces particle assignments yielding reference-quality reconstructions in fewer epochs. We demonstrate how to learn representations which capture biological information for use across multiple cryo-EM inverse problems.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
representation learning, cryo-em

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kr3JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper frames core cryo-EM tasks, including pose estimation, heterogeneity analysis, population estimation, and reconstruction, as inverse problems over a shared image-formation process. It argues that they should therefore benefit from a common particle-level representation. It further argues that the 2D augmentations standard in self-supervised learning do not correspond to physically realizable variation in cryo-EM imaging. Instead, it constructs training views by rendering the same structure through the cryo-EM forward model under different poses and imaging conditions.

The authors introduce EMber, a family of vision transformers ranging from 21M to 1.1B parameters, trained with DINOv3 self-distillation and an iBOT masked-patch objective on simulated projections combined with experimental particles. The evaluation spans k-NN dataset discrimination, compositional classification on experimental CAHRA data, fine-tuned conformational classification on simulated Hsp90, linear probe population estimation on SARS-CoV-2 Spike, and particle grouping within CryoDRGN-AI for heterogeneous reconstruction. Across these tasks, the paper reports consistent gains over the particle foundation models Cryo-IEF and CryoUNI.

Critical strengths
  • Physically grounded view construction at scale. The paper makes a well-reasoned case that, for cryo-EM, positive pairs should be generated by rendering the same structure through the image-formation forward model rather than by applying 2D augmentations that do not correspond to physically realizable variation (Sec. 4, Fig. 2). Importantly, the authors engineered the renderer to run inside the training loop, making this approach practical at the scale of tens of millions of particles.

  • Breadth and framing of the evaluation. Rather than relying on dataset-ID kNN, which the authors acknowledge is confounded by acquisition conditions (Sec. 6.1), each evaluation is designed to isolate a specific factor. CAHRA fixes imaging conditions so that accuracy reflects molecular composition (Sec. 6.1), and the probe comparison tests whether global pooling discards structural information (Fig. 3C). Population estimation targets the ensemble-level quantity of biological interest (Sec. 6.3), while the two-phase reconstruction protocol separates particle grouping from pose error (Sec. 6.4). The authors also report where the method fails, such as near-chance accuracy at SNR 0.001 (Fig. 4C).

Critical weaknesses
  1. The pretraining data likely overlap with the evaluation structures, and most evaluations are simulator-to-simulator. Both favor EMber over the baselines and weaken the claim that it transfers “across proteins.”

    • The simulated pretraining set contains about 268k PDB assemblies (Sec. 5). Most structural evaluations use well-studied PDB systems: Hsp90 (Secs. 6.2, 6.4), the SARS-CoV-2 Spike up/down states (Sec. 6.3), and the eight TomoTwin structures (Sec. 6.4). These, or close homologs, are likely present in the pretraining set. If so, EMber has already seen renderings of these exact molecules with exact structural correspondence, whereas Cryo-IEF and CryoUNI have not. In addition, these evaluation images appear to be rendered with the same forward model used for pretraining (Fig. 4A; Sec. 6.3), so the baselines face a domain shift that EMber does not. A similar concern applies to Fig. 6: pretraining pairs particles from the same EMPIAR entry (Sec. 5), which directly optimizes dataset discrimination, and the 29 CryoPPP datasets may overlap with the 214 pretraining entries. Notably, the largest reported gains (population estimation and reconstruction) come from settings where these confounds apply, while on CAHRA, the only unseen experimental benchmark, the margins are smaller.
  2. The proposed design choices are not isolated, so the reported gains cannot be attributed to the paper’s contributions.

    • The paper motivates several specific design choices: rendering views through the forward model instead of applying 2D augmentations (Sec. 4), self-distillation instead of a contrastive objective (Secs. 1, 4), the iBOT masked-patch loss for dense features (Abstract, Sec. 4), and the simulated-to-experimental data curriculum (Sec. 5). None of these is ablated in the main text under fixed data and architecture. Meanwhile, EMber differs from Cryo-IEF and CryoUNI simultaneously in pretraining data, objective, view construction, and model scale, so the comparisons conflate all of these factors. The one mechanism-level finding, that spatial readouts outperform pooled ones (Fig. 3C), holds for every backbone, and the authors themselves note that it does not credit their patch-level objective (Sec. 6.1). As a result, the paper’s main methodological takeaway, that the forward model should be used to construct supervision (Sec. 7), remains asserted rather than demonstrated.
Questions for authors
  • Which evaluation molecules (Hsp90, the Spike up/down states, the eight TomoTwin structures, the CAHRA proteins) or homologs (e.g., >30% sequence identity) are in the 268k pretraining assemblies? How many of the 29 CryoPPP EMPIAR entries are among the 214 pretraining entries? Can the authors report results with these held out?
  • Could the authors provide a small-scale ablation (e.g., ViT-S on a reduced data budget) comparing forward-model rendered views against 2D augmentations applied to rendered images, with all else fixed, evaluated on CAHRA and Hsp90? This would directly test the paper’s central methodological claim (Sec. 4). If compute allows, ablating the iBOT loss and the experimental-data curriculum would further clarify which components drive the gains.
  • Which comparisons produce the abstract’s “14.85 / 14.26 pp” figures? From the plots, they appear to compare EMber-g (1.1B) with ViT-B baselines, which is not “comparable model sizes” as stated in Sec. 6.1.
  • In Sec. 6.4, is k in k-means set to the true number of states? The text says only EMber recovers the open Hsp90 state. But Fig. 7B seems to show Cryo-IEF’s closed-state reconstruction (7.42 Å) as worse than its open-state one (6.08 Å). Please reconcile these.
Reviewer_duHwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents EMber, a family of pretrained vision transformers that learn general-purpose representations from cryo-EM particle images. Instead of relying only on standard image-space augmentations, the authors use the cryo-EM forward model to generate physically meaningful self-supervised views. Training follows the DINOv3 recipe, combining self-distillation with an iBOT masked-patch objective. The pretraining data include simulated projections of roughly 268,000 molecular assemblies and more than 50 million experimental particles drawn from 214 EMPIAR entries.

EMber is evaluated on molecular composition classification, conformational discrimination, conformational population estimation, and particle grouping for heterogeneous reconstruction. Across these tasks, the authors report gains over several existing cryo-EM representation models, which suggests the pretrained features transfer across different structural inference problems.

Critical strengths
  1. A pretraining framework built around the cryo-EM imaging process.

Generating self-supervised views through the cryo-EM forward model is a well-motivated choice. Generic image-space augmentations cannot capture the main sources of variation in particle images. Simulation can vary orientation, CTF parameters, and noise directly, while the underlying structure stays fixed for each synthetic pair. This gives the model an inductive bias that matches what a molecular representation should be invariant to.

  1. Transfer to tasks beyond standard classification.

The evaluation includes conformational population estimation and particle grouping for heterogeneous reconstruction, which are closer to real structural-biology workflows than classification benchmarks. On population estimation, EMber’s estimates track the ground truth more closely than those of the compared representation models. On Hsp90 reconstruction, clustering EMber features gives resolutions close to the oracle after two epochs (4.53/4.55 Å vs. 4.48/4.50 Å). Together, these results suggest that a shared pretrained representation can be useful in practice.

Critical weaknesses
  1. The contribution of the proposed pretraining strategy is not isolated by controlled ablations.

The paper’s main methodological contribution has three parts: forward-model-based view construction, a self-distillation objective, and a staged schedule for mixing synthetic and experimental data. The experiments do not separate the effect of each one.

Section 4 argues that standard image-space augmentations can impose physically inappropriate invariances, which motivates building views through the forward model. The argument is reasonable, but it is never tested directly. There is no comparison between physics-based and standard augmentations using the same backbone, training data, and compute budget, so it is unclear how much of the reported gain comes from the view construction itself.

The same applies to the other design choices. The paper does not systematically evaluate the iBOT masked-patch objective, the relative contributions of synthetic and experimental data, or the data-mixing schedule. The supplementary material documents these procedures in detail but includes no ablations for them.

This matters because the gains over existing pretrained models could also come from differences in training data, model capacity, or optimization budget. Without matched comparisons, the current results do not show that the domain-adapted pretraining strategy is responsible for the improvements. Since these design choices are the paper’s central methodological contribution, I see this as a significant limitation.

  1. Limited evidence of generalization across molecular systems and realistic inverse-problem settings.

The paper positions EMber as a general-purpose representation that can be reused across molecular systems and inverse problems. Yet several of the main conclusions rest on a small number of systems evaluated in controlled settings. Conformational classification is tested mainly on Hsp90, population estimation on SARS-CoV-2 Spike, and heterogeneous reconstruction on TomoTwin-8 and Hsp90. These results show promising transfer, but they say little about robustness across diverse structures, heterogeneous states, and imaging conditions.

The pose-probing experiments in the supplementary material point to a related limitation. Pose is decodable with protein-specific probes, but accuracy drops toward chance when a single probe is shared across many proteins. How general the representation is may therefore depend on the downstream task and the readout.

The reconstruction protocol is also favorable to the method. The final RELION reconstruction uses ground-truth poses, the number of clusters is set to the true number of structures, and clusters are assigned to references by an exhaustive search for the pairing with the best mean resolution. This setup is reasonable for isolating clustering quality, but it does not show how EMber would perform in a realistic setting where poses and the number of states are unknown.

In short, the experiments provide evidence of useful transfer, but they do not yet support the broader claim of a general-purpose cryo-EM representation.

Questions for authors

Q1. Can the authors ablate the main pretraining components? With the backbone and compute budget held fixed, how does forward-model-based view construction compare with standard image-space augmentation? What do iBOT, the synthetic and experimental data, and the staged data-mixing schedule each contribute? These results would substantially affect my assessment of the methodological contribution.

Q2. How well do the representations hold up across molecular systems and in less constrained reconstruction settings? Can the authors provide further evidence of cross-protein generalization, and explain what the pose-probing results in the supplementary material imply for downstream use? For heterogeneous reconstruction, how would the results change if poses were estimated and the number of states were unknown, instead of using oracle information as in the current evaluation?

Q3. Could the authors clarify the protocol for EMPIAR entry classification, and whether evaluation entries overlap with pretraining entries? According to Appendix C.1, k is selected on the same query set used to report final accuracy. Why was a separate validation split not used, and how much optimistic bias might this introduce? In addition, are any of the 29 CryoPPP entries used for evaluation also among the 214 EMPIAR entries used for pretraining? If so, how should the classification results be interpreted?

Q4. Does same-entry positive pairing suppress conformational information? Section 5 and Appendix B.3 state that, with probability 0.5, a particle is paired with a different particle from the same EMPIAR entry as a positive view, even though the two may differ in conformation or composition. This could push the model toward invariance to exactly the structural differences it is meant to preserve. How does conformational discrimination change when this pairing is disabled, or when its probability is varied?

Q5. What does pretraining cost, and how does performance scale with compute? EMber ranges from about 21M to 1.1B backbone parameters and is trained for 265k steps with online CryoJAX rendering, but the paper reports only inference FLOPs. Could the authors report total GPU hours, peak memory, and the overhead of online simulation? It would also help to show how downstream performance scales with pretraining compute. These numbers matter for judging how practical the approach is for other groups to adopt or extend.

Reviewer_dPasOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes to pre-train a cryo-EM model, EMber, using both large-scale simulation of cryo-EM data from a large protein database (OpenProteinSet) and experimental data scraped from EMPIAR. EMber is trained using the DINOv3 recipe. EMber is evaluated on conformation classification and clustering.

Critical strengths

(S1) Use of simulated and experimental data: The use of simulated data in addition to experimental data potentially allows much greater training data scale and the training of larger models.

(S2) Cryo-EM models trained at much greater scale than before: EMber was able to scale beyond comparable models CryoIEF and CryoUNI to ViT-L/G.

(S3) Adds another major SSL framework to the already existing evaluated frameworks: EMber evaluates DINO-based SSL models, while previous work CryoIEF and CryoUNI used contrastive and MAE-based approaches, respectively.

Critical weaknesses

(W1) The proteins in the experiments likely all appear in EMber’s training data but some of the proteins likely do not appear in CryoIEF’s and CryoUNI’s: CryoPPP overlaps significantly with CryoIEF’s training set while CryoUNI has ~30% overlap. EMber is likely exposed to all CryoPPP proteins through OpenProteinSet and to at least some of the CryoPPP EMPIAR entries through the scraped EMPIAR portion. The six CAHRA heterogeneity challenge, HSP90, and the Covid spike protein have all been seen in EMber’s training data. For CryoIEF and CryoUNI/CryoCRAB, the situation is mixed. For CAHRA, GDH, ADH, amylase and globulin are likely unseen by CryoUNI and CryoIEF, while aldolase and PKM2 are seen. This comes with the caveat that CryoIEF did not release the identity of 50 in-house datasets. For HSP90, the bound state has been seen by CryoUNI, whereas multiple states have likely been seen by EMber through simulating cryo-EM images from OpenProteinSet. The Covid spike protein is in every model’s training data.

The major weakness resulting from this is confounded comparisons. For CAHRA and HSP90, the fact that not all baselines have seen the protein during training makes the comparison not apples-to-apples and gives EMber an advantage. This advantage is possibly increased by the use of a small number of fine-tuning steps, such as only 10K steps for HSP90 and only ~1K steps for CAHRA, which favors methods that have already seen the protein during pre-training.

Additionally, the current baseline for the conformation classification results, a ResNet-18, is not the best comparison because it is both not the same type of architecture and has significantly fewer parameters than even a ViT-S. A better and more fair comparison would be supervised ViT-S/B/L/G.

(W2) No ablation study: EMber uses DINOv3’s training recipe. There is no ablation study of whether the different components of DINOv3’s training recipe are actually effective for cryo-EM. There is also no ablation study of which augmentations were most effective for experimental data or whether unphysical augmentations actually hurt performance. There is also no study of how much the data scale or a mix of simulation data and experimental data helps EMber.

(W3) Claims of downstream effectiveness require more support: While the paper is not completely clear on the setup of the heterogeneous ab initio reconstruction experiment, my understanding is that for all methods, cryoDRGN-AI was used as the base model. What was compared was different choices for the conformation encoder: a lookup table (cryoDRGN-AI’s default), a CNN, and frozen EMber, CryoIEF, and CryoUNI encoders with a learnable adapter. I think this setup has a few flaws: first, full cryoDRGN-AI models are not trained or evaluated, so this experiment is not demonstrating downstream success in ab initio reconstruction. Second, learning the poses is an important part of ab initio reconstruction, but the evaluation protocol is at best an indirect and weak way to evaluate them. As discussed in W1, the differing exposure to proteins during pre-training also confounds the results. I argue that the results reflect this. In the HSP90 reconstruction experiment, the CryoIEF model, which has not seen HSP90 during pre-training as far as I can tell, performs the worst out of all neural network encoders, including the CNN with no pre-training. CryoUNI and EMber, who have seen HSP90 during pre-training, are clearly better than the other two methods. Additionally, as discussed below, a lightweight attentive probe is probably more fair than a convolution-based probe. Because the amount of training is so short (1-3 epochs), the experiment favors fast-converging models. CryoDRGN-AI is usually trained for 100+ epochs. It’s also my experience that even multiple runs of the same ab initio reconstruction algorithm can have wildly different convergence speeds, so the apparent advantage of EMber may be due to a lucky run that has fast convergence. This means that multiple training runs should be carried out and aggregate results reported.

(W4) Linear probe results disfavor CryoUNI: It is well-known that linear probes favor contrastive methods such as CryoIEF and DINO-based methods such as EMber and disfavor MAE-based methods such as CryoUNI. I believe that this is also supported by the paper’s own results, as CryoUNI has a big jump with a convolutional probe whereas the other two methods see less of an improvement, and that the gap between CryoUNI and the other methods is less in the fine-tuning regime, as predicted by prior literature. One better way may be to use a lightweight attentive probe [1] in addition to the already provided probes.

(W5) Differences in imaging acquisition hardware may advantage EMber: The authors acknowledge that acquisition-specific factors may play a role in the results in the discussion of the CryoPPP results. However, I believe that there is a possibility that it also affects the CAHRA results as well. Since CAHRA was imaged using relatively recent technology (Selectris, Falcon 4i, Cs corrector), EMber’s experimental data, which was scraped with a 2026 cutoff, is much more likely to include datasets with the Selectris/Falcon 4i/Cs corrector than CryoIEF or CryoUNI/CryoCRAB, which date to 2024. At that time, Selectris and Falcon 4i were still new and so it’s likely that little of the data in CryoIEF/CryoCRAB was collected with those two pieces of hardware.

The weaknesses are roughly ranked by importance to my recommendation.

[1] Psomas, Bill, et al. "Attention, please! revisiting attentive probing through the lens of efficiency." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors
  1. How much does having seen the protein during pre-training help? E.g. For CAHRA, are CryoIEF and CryoUNI better on aldolase and PKM2, which were seen during pre-training, than the other proteins, which were likely not?
  2. How do probe results change when we include the lightweight attentive probe described earlier?
  3. How does EMber perform when tested on proteins it was not exposed to during pre-training?
  4. How do the methods rank in an equal-data setting, tested on both proteins seen during training and heldout proteins?
  5. Do the improvements from DINO to DINOv2, or DINOv2 to DINOv3, actually help in the cryo-EM setting?
  6. How much advantage does using simulated and experimental data give over using just experimental data or just simulated data?
  7. How does the pose accuracy compare between EMber and the baselines for the heterogeneous ab initio experiments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.