- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper observes that existing cryo-EM representation models are each specialized to a single observation type, such as micrographs, SPA particles, or reconstructed density maps, and asks whether a single encoder can learn representations that transfer across the different 2D observations produced before reconstruction in both SPA and cryo-ET. It introduces CryoVERSE, a shared vision transformer jointly pretrained on roughly 144 million SPA micrographs, SPA particles, cryo-ET tilt images and tilt-derived cryo-ET particle views. To handle inputs of different spatial extent within one backbone, the model uses flexible positional encoding and input-specific view generation. Pretraining combines momentum contrastive learning with a pose-aware extension, in which distinct particles with nearby viewing directions serve as down-weighted positives, together with masked patch-level feature prediction on wide-field images to preserve spatially resolved features. The model is evaluated on SPA pose and structure classification, particle quality ranking and quality-guided reconstruction, micrograph particle picking, and cryo-ET particle-type and tilt-angle probes, mainly in comparison with Cryo-IEF. The authors claim consistent improvements, that broader pretraining does not dilute particle-level information, and that joint pretraining across observation types and scales yields transferable representations.
Critical strengths
-
First joint SPA-cryo-ET pretraining at scale, with a nontrivial cryo-ET data pipeline. To my knowledge, this is the first encoder jointly pretrained on 2D observations from both SPA and cryo-ET, drawing on a corpus of 143.7M images. The cryo-ET construction pipeline (App. B) traces each particle annotation through the CryoET Data Portal provenance graph to a consistent tomogram, tilt series, and alignment record, then projects the 3D annotations back into the acquired tilt images. This is nontrivial engineering, and it would be community asset in its own right if the resulting particle-view corpus and dataset list were released by the authors. On the empirical side, CryoVERSE outperforms Cryo-IEF on pose classification for all 12 simulated datasets (Table 7), with the largest gains on the three smallest targets, where pose assignment is practically hardest.
-
A principled, geometry-aware contrastive objective. The pose-aware positive reuses the refined poses that SPA processing already produces, pairing distinct particles with nearby viewing directions as positives. Symmetry-aware angular distance (Eq. 7) and smooth distance-based down-weighting (Eq. 11) keep these cross-particle pairs from being treated as reliably as same-image pairs. This brings cryo-EM projection geometry into self-supervised learning without requiring new annotations. The ablations support the intended mechanism.
Critical weaknesses
W1. The central claim, transfer from joint pretraining across observation types and scales, is weakly supported.
- Gains over Cryo-IEF largely survive removing cross-scale data. The abstract presents the gains over Cryo-IEF as evidence that joint pretraining yields transferable representations, but most of the 4.53 pp pose gain persists without wide-field data or without any cryo-ET data (Table 2). The headline gains thus likely reflect recipe changes, data scale, and pose-aware positives (which add pose supervision that Cryo-IEF lacks) rather than primarily joint pretraining. A SPA-particles-only variant trained with the same recipe would help isolate the contribution of joint pretraining.
- No SPA to cryo-ET transfer beyond generic pretraining. Without cryo-ET data, type-macro accuracy matches natural-image MAE and MoCo v3, and mean particle accuracy is lower (Table 3). A cryo-ET-only variant is needed to test whether SPA data helps cryo-ET at all.
- Dense-transfer evidence is marginal. In particle picking, CryoVERSE’s F1 exceeds a fine-tuned ImageNet MoCo v3 by only 0.016, and removing MIM and registers, the components introduced to strengthen dense features, has mixed effects (AUPRC 0.535 vs. 0.529; F1 0.520 vs. 0.543; Fig. 4).
- No variance is reported for any result, so it is unclear whether these small effects exceed run-to-run variation.
W2. Several evaluation results are confounded or internally inconsistent.
- The tilt-angle probe is susceptible to shortcuts via same-series neighbors. The 5-NN angle regression excludes only the query image (App. E.5.2), so adjacent tilts of the same series, which show the same specimen field, remain available as neighbors and can make the task near-trivial. The authors exclude same-particle views for the cryo-ET particle probe (App. E.5.3) but not here. Leave-one-tilt-series-out retrieval and simple intensity-statistic baselines are needed before the MAE gap (Table 9) can be read as geometry encoding.
- Reconstruction results are inconsistent across Fig. 3c and Table 8. For the top-20% CryoRanker-VERSE subsets, Fig. 3c reports 2.99 / 3.34 / 2.72 / 2.94 Å on EMPIAR-10250 / 10405 / 10556 / 10560, whereas Table 8 reports 3.13 / 3.29 / 2.72 / 2.93 Å. With the Fig. 3c value, CryoRanker-VERSE is worse than CryoRanker-IEF on EMPIAR-10405 (3.34 vs. 3.29 Å). I believe this discrepancy is nontrivial, as the reconstruction experiment is the paper’s most direct evidence of practical benefit, and the two sources disagree on whether CryoVERSE improves over Cryo-IEF on EMPIAR-10405.
- Specimen-level overlap in cryo-ET can’t be ruled out. Specimen-level exclusion is stated for the SPA evaluation sets but not for cryo-ET (App. A.4), and the cryo-ET pretraining collections are reported only in aggregate. CZI-10468/10469 come from the same CZII phantom program as the challenge datasets used to illustrate particle construction (Fig. 5), and the largest per-species gains fall on the shared virus-like capsid (Table 11). Because Table 3 shows that the cryo-ET gains depend on in-domain cryo-ET pretraining data, this overlap determines whether those results reflect generalization.
Questions for authors
-
Please reconcile Fig. 3c (top-quintile resolutions 2.99 / 3.34 / 2.72 / 2.94 Å) with Table 8 (3.13 / 3.29 / 2.72 / 2.93 Å) for the same four datasets. Were these different refinement runs or protocols?
-
What fraction of pretraining particle datasets passed the Å threshold and therefore used pose-based positives (App. A.1.2)? How does the pose-aware ablation (Table 2) look when computed only over models trained without any pose-derived positives?
-
In the tilt-angle probe, adjacent tilts from the same series remain available as neighbors (App. E.5.2). Could the authors report results with leave-one-tilt-series-out retrieval, alongside a simple baseline using image intensity statistics? If CryoVERSE retains a clear advantage over DRACO under this protocol, it would support the claim that the representations encode projection geometry.
-
To isolate the contribution of joint pretraining, could the authors add (i) a SPA-particles-only variant trained with the CryoVERSE recipe and (ii) a cryo-ET-only variant?
-
Were the ablated pretraining variants compute-matched to the full model in terms of steps and images seen? Removing wide-field or cryo-ET batches changes the training stream unless compensated.
-
SPA pose and structure classification (Sec. 4.1) are evaluated only on simulated particles, where structure accuracy is already near ceiling (96.94% → 98.94%) and the largest pose gains on small targets may not reflect realistic noise, CTF, and ice conditions. Could the authors evaluate pose organization on a few real SPA datasets held out from pretraining, using poses from high-quality refinements? Gains of similar size on real data, especially for small targets, would substantially strengthen Sec. 4.1.