acceptodds
Under review as a conference paper at ICLR 2027

CryoVERSE: A Foundation Model across Cryo-EM Observation Types and Scales

Abstract

Cryogenic electron microscopy (cryo-EM) reconstructs three-dimensional macromolecular structures from noisy two-dimensional observations spanning diverse data types and spatial scales. Existing representation-learning approaches, however, typically specialize in individual observation types, limiting knowledge transfer across single-particle analysis (SPA) and cryogenic electron tomography (cryo-ET) workflows. We introduce CryoVERSE (Cryo-EM Versatile Encoder for Representations across Scales), a unified foundation model jointly pretrained on four pre-reconstruction 2D cryo-EM observation types: SPA micrographs, SPA particles, cryo-ET tilt images, and tilt-derived cryo-ET particles. CryoVERSE uses a scale-flexible shared encoder with pose-aware contrastive learning and masked patch-level prediction to capture transferable global and spatial representations across these observation types and scales. Across diverse downstream evaluations, CryoVERSE improves molecular structure classification, projection-pose classification, and particle quality ranking, transfers effectively to dense particle localization in micrographs, and preserves structural identity and acquisition-angle information in cryo-ET. These results demonstrate that joint pretraining across cryo-EM observation types and scales can yield transferable global and spatial representations within a shared backbone, providing a unified representation basis for diverse pre-reconstruction cryo-EM analysis tasks.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
cryo-electron microscopy, foundation models, self-supervised learning, contrastive learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XjAjOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper observes that existing cryo-EM representation models are each specialized to a single observation type, such as micrographs, SPA particles, or reconstructed density maps, and asks whether a single encoder can learn representations that transfer across the different 2D observations produced before reconstruction in both SPA and cryo-ET. It introduces CryoVERSE, a shared vision transformer jointly pretrained on roughly 144 million SPA micrographs, SPA particles, cryo-ET tilt images and tilt-derived cryo-ET particle views. To handle inputs of different spatial extent within one backbone, the model uses flexible positional encoding and input-specific view generation. Pretraining combines momentum contrastive learning with a pose-aware extension, in which distinct particles with nearby viewing directions serve as down-weighted positives, together with masked patch-level feature prediction on wide-field images to preserve spatially resolved features. The model is evaluated on SPA pose and structure classification, particle quality ranking and quality-guided reconstruction, micrograph particle picking, and cryo-ET particle-type and tilt-angle probes, mainly in comparison with Cryo-IEF. The authors claim consistent improvements, that broader pretraining does not dilute particle-level information, and that joint pretraining across observation types and scales yields transferable representations.

Critical strengths
  • First joint SPA-cryo-ET pretraining at scale, with a nontrivial cryo-ET data pipeline. To my knowledge, this is the first encoder jointly pretrained on 2D observations from both SPA and cryo-ET, drawing on a corpus of 143.7M images. The cryo-ET construction pipeline (App. B) traces each particle annotation through the CryoET Data Portal provenance graph to a consistent tomogram, tilt series, and alignment record, then projects the 3D annotations back into the acquired tilt images. This is nontrivial engineering, and it would be community asset in its own right if the resulting particle-view corpus and dataset list were released by the authors. On the empirical side, CryoVERSE outperforms Cryo-IEF on pose classification for all 12 simulated datasets (Table 7), with the largest gains on the three smallest targets, where pose assignment is practically hardest.

  • A principled, geometry-aware contrastive objective. The pose-aware positive reuses the refined poses that SPA processing already produces, pairing distinct particles with nearby viewing directions as positives. Symmetry-aware angular distance (Eq. 7) and smooth distance-based down-weighting (Eq. 11) keep these cross-particle pairs from being treated as reliably as same-image pairs. This brings cryo-EM projection geometry into self-supervised learning without requiring new annotations. The ablations support the intended mechanism.

Critical weaknesses

W1. The central claim, transfer from joint pretraining across observation types and scales, is weakly supported.

  • Gains over Cryo-IEF largely survive removing cross-scale data. The abstract presents the gains over Cryo-IEF as evidence that joint pretraining yields transferable representations, but most of the 4.53 pp pose gain persists without wide-field data or without any cryo-ET data (Table 2). The headline gains thus likely reflect recipe changes, data scale, and pose-aware positives (which add pose supervision that Cryo-IEF lacks) rather than primarily joint pretraining. A SPA-particles-only variant trained with the same recipe would help isolate the contribution of joint pretraining.
  • No SPA to cryo-ET transfer beyond generic pretraining. Without cryo-ET data, type-macro accuracy matches natural-image MAE and MoCo v3, and mean particle accuracy is lower (Table 3). A cryo-ET-only variant is needed to test whether SPA data helps cryo-ET at all.
  • Dense-transfer evidence is marginal. In particle picking, CryoVERSE’s F1 exceeds a fine-tuned ImageNet MoCo v3 by only 0.016, and removing MIM and registers, the components introduced to strengthen dense features, has mixed effects (AUPRC 0.535 vs. 0.529; F1 0.520 vs. 0.543; Fig. 4).
  • No variance is reported for any result, so it is unclear whether these small effects exceed run-to-run variation.

W2. Several evaluation results are confounded or internally inconsistent.

  • The tilt-angle probe is susceptible to shortcuts via same-series neighbors. The 5-NN angle regression excludes only the query image (App. E.5.2), so adjacent tilts of the same series, which show the same specimen field, remain available as neighbors and can make the task near-trivial. The authors exclude same-particle views for the cryo-ET particle probe (App. E.5.3) but not here. Leave-one-tilt-series-out retrieval and simple intensity-statistic baselines are needed before the MAE gap (Table 9) can be read as geometry encoding.
  • Reconstruction results are inconsistent across Fig. 3c and Table 8. For the top-20% CryoRanker-VERSE subsets, Fig. 3c reports 2.99 / 3.34 / 2.72 / 2.94 Å on EMPIAR-10250 / 10405 / 10556 / 10560, whereas Table 8 reports 3.13 / 3.29 / 2.72 / 2.93 Å. With the Fig. 3c value, CryoRanker-VERSE is worse than CryoRanker-IEF on EMPIAR-10405 (3.34 vs. 3.29 Å). I believe this discrepancy is nontrivial, as the reconstruction experiment is the paper’s most direct evidence of practical benefit, and the two sources disagree on whether CryoVERSE improves over Cryo-IEF on EMPIAR-10405.
  • Specimen-level overlap in cryo-ET can’t be ruled out. Specimen-level exclusion is stated for the SPA evaluation sets but not for cryo-ET (App. A.4), and the cryo-ET pretraining collections are reported only in aggregate. CZI-10468/10469 come from the same CZII phantom program as the challenge datasets used to illustrate particle construction (Fig. 5), and the largest per-species gains fall on the shared virus-like capsid (Table 11). Because Table 3 shows that the cryo-ET gains depend on in-domain cryo-ET pretraining data, this overlap determines whether those results reflect generalization.
Questions for authors
  • Please reconcile Fig. 3c (top-quintile resolutions 2.99 / 3.34 / 2.72 / 2.94 Å) with Table 8 (3.13 / 3.29 / 2.72 / 2.93 Å) for the same four datasets. Were these different refinement runs or protocols?

  • What fraction of pretraining particle datasets passed the <4.2<4.2 Å threshold and therefore used pose-based positives (App. A.1.2)? How does the pose-aware ablation (Table 2) look when computed only over models trained without any pose-derived positives?

  • In the tilt-angle probe, adjacent tilts from the same series remain available as neighbors (App. E.5.2). Could the authors report results with leave-one-tilt-series-out retrieval, alongside a simple baseline using image intensity statistics? If CryoVERSE retains a clear advantage over DRACO under this protocol, it would support the claim that the representations encode projection geometry.

  • To isolate the contribution of joint pretraining, could the authors add (i) a SPA-particles-only variant trained with the CryoVERSE recipe and (ii) a cryo-ET-only variant?

  • Were the ablated pretraining variants compute-matched to the full model in terms of steps and images seen? Removing wide-field or cryo-ET batches changes the training stream unless compensated.

  • SPA pose and structure classification (Sec. 4.1) are evaluated only on simulated particles, where structure accuracy is already near ceiling (96.94% → 98.94%) and the largest pose gains on small targets may not reflect realistic noise, CTF, and ice conditions. Could the authors evaluate pose organization on a few real SPA datasets held out from pretraining, using poses from high-quality refinements? Gains of similar size on real data, especially for small targets, would substantially strengthen Sec. 4.1.

Reviewer_g7jiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CryoVERSE is a unified representation model for four types of pre-reconstruction cryo-EM observations: SPA particles, SPA micrographs, cryo-ET tilt images, and tilt-derived particles. The model was evaluated in detail on downstream tasks in cryo-electron microscopy, and the results demonstrate that the pre-trained model effectively improves performance on these tasks.

Critical strengths
  1. The paper addresses a meaningful unification problem in cryo-EM representation learning. Images produced by different cryo-electron microscopes vary in their characteristics; using a unified model allows for more effective processing of these diverse images.
  2. Experimental results and ablation studies demonstrate the effectiveness of this pre-trained model.
  3. The model operates directly on pre-reconstruction 2D observations, making it useful early in the cryo-EM processing workflow. Its representations can support particle localization and quality assessment before a 3D reconstruction is available. This provides a practical advantage over approaches that require reconstructed density maps as input.
Critical weaknesses
  1. The methodological novelty is modest. Much of the framework combines established representation-learning techniques. The experiments establish the model’s utility more clearly than they establish broader insights into when and why joint learning across cryo-EM observation types is beneficial. This limits the general methodological significance of the work.
  2. The advantages of a unified encoder over a collection of specialized encoders remain insufficiently explored. An alternative design is to use observation-specific encoders, with inputs assigned to the appropriate model either by their known type or through a routing mechanism. A routing-based system could potentially accommodate a new cryo-EM observation type by training an additional specialized encoder and incorporating it into the existing system. In contrast, extending a unified encoder may require further joint training and validation to preserve its performance on existing observation types. It is therefore unclear whether the unified approach offers an advantage in the time and computational cost of incorporating new data types. A comparison of these adaptation costs, together with performance retention on existing tasks, would help clarify the trade-offs between the two designs.
  3. In the pre-training dataset, the data undergoes processes such as particle selection, 2D classification, and reconstruction. This biases the model toward handling these relatively "easier" samples. However, is the model also capable of effectively processing "difficult" samples—those characterized by a low signal-to-noise ratio and high heterogeneity? The authors need to explore this question in greater depth.
Questions for authors

See Strengths and Weaknesses

Reviewer_nBAtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CryoVERSE pretrains one ViT-B/14 encoder (RoPE + registers) on 143.7M 2D cryo-EM observations of four types: SPA micrographs, SPA particles, cryo-ET tilt images, and cryo-ET particles. Its objective is MoCo-style contrastive learning with "pose-aware" positives (distinct particles with nearby viewing directions, weighted by angular distance), plus iBOT-style masked prediction on wide-field images. The paper evaluates on SPA pose and structure kNN, particle-quality ranking with reconstructions, micrograph particle picking, and cryo-ET particle-type and tilt-angle kNN. It reports consistent gains over Cryo-IEF and other baselines, with ablations over data types and objective components.

Critical strengths
  1. One shared encoder, evaluated broadly with controlled ablations. The paper evaluates a single backbone on global, dense, and cryo-ET tasks, and its ablations remove one data type or objective component at a time. These are the right controls for the central claim of cross-type pretraining, which makes the paper a useful empirical reference.
  2. A domain-grounded objective, validated downstream in a way that matters in practice. Pose-aware positives build projection geometry into contrastive learning, and the ablation shows they mainly help pose organization, as intended. The quality evaluation goes beyond proxy metrics to actual reconstructions, which is what practitioners care about.
Critical weaknesses
  1. The in-family gains are small and reported without variance. On the key ablations, the full model barely beats the variant trained without wide-field data (structure 98.94 vs. 98.70; pose 95.67 vs. 94.83). On picking, its AUPRC is lower than the variant without MIM and registers (0.529 vs. 0.535). All results appear to be single runs on 4–12 datasets, with no seeds or confidence intervals, so many of the differences cannot be told apart from noise.
  2. The comparison with Cryo-IEF is confounded. CryoVERSE changes data scale, architecture (patch size 14, RoPE, registers), and objective at the same time. The improvement therefore cannot be attributed to cross-type pretraining, which is the paper's central claim.
  3. The baselines are weak or under-specified. The MAE/MoCo v3 baselines (9.5% pose, 30.1% structure) look like natural-image checkpoints, not generic SSL trained on the same cryo-EM corpus. For picking, it is unclear whether the baselines were retrained on the same accessions, or how detection thresholds were chosen for each method.
  4. Leakage and circularity are not ruled out. The main text does not show accession- or protein-level exclusion between the large pretraining corpus and the evaluation datasets. Pretraining particles were curated with CryoRanker and paired using cryoSPARC poses, so pose-aware positives amount to weak supervision tied to what the evaluations measure. In tilt-angle kNN, neighbors from the same tilt series could make the task nearly trivial.
  5. Methodological novelty is limited, and the core method appears only in the appendix. The building blocks (MoCo v3, iBOT, KoLeo, RoPE, registers, neighbor positives) already exist. The main new piece is the pose-aware positive, yet its loss, weighting function, threshold, and symmetry handling are described only in Appendix C.2.
Questions for authors

Clarifications (these could change my assessment):

Variance. Can you report mean ± std over multiple seeds, or bootstrap CIs, for the key ablation deltas: full vs. w/o wide-field, and full vs. w/o MIM+registers?

Leakage. Are the evaluation datasets excluded from pretraining at the accession level, the protein level, or both? This covers the 12 simulated targets, the CryoRanker and reconstruction datasets, CryoPPP, and CZI. In tilt-angle kNN, are neighbors from the same tilt series excluded?

Baselines. Were MAE/MoCo v3 trained on natural images or on your corpus? Were the picking baselines retrained on the same accessions, and how were their thresholds chosen?

Pose-aware positives. What fraction of pairs use cross-particle positives rather than the same-image fallback? How are symmetry and in-plane rotation handled?

Reconstruction. How does the top-20% subset compare with the full particle set and with standard 2D-classification curation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.