acceptodds
Withdrawn from ICLR 2027

CryoGaussian: A Unified Gaussian Splatting Method for Homogeneous and Heterogeneous Cryo-EM Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

12 views

Abstract

Single-particle cryo-electron microscopy (cryo-EM) resolves biomolecular structures at near-atomic resolution without crystallization. Reconstructing a biomolecule's 3D density from noisy 2D particle images presents two core challenges: (1) recovering as much reliable structural detail as the data support in both homogeneous and heterogeneous reconstruction; and (2) identifying rare and transient compositional states and tracing subtle, continuous conformational changes. To address both challenges, we propose CryoGaussian, a cryo-EM-native Gaussian splatting method that unifies homogeneous and heterogeneous reconstruction for high-fidelity structural recovery. Starting from randomly initialized Gaussians, CryoGaussian fits an explicit 3D density representation directly to particle images through orthographic line-integral projection modulated by the contrast transfer function. To model heterogeneity, CryoGaussian learns particle-image and per-Gaussian latents that jointly condition Gaussian attributes to capture compositional and conformational variation. The inferred states then guide optimization of the Gaussian representation for detailed per-state reconstruction. We validate CryoGaussian: (1) on all five synthetic CryoBench datasets, it achieves competitive performance in heterogeneity estimation and per-state reconstruction; (2) on homogeneous experimental data, it resolves structural detail in the RAG1-RAG2 complex, the SARS-CoV-2 spike on intact virions, and the 80S ribosome; and (3) on heterogeneous experimental data, it reveals previously validated structural features through unsupervised heterogeneity estimation, including a 50S assembly intermediate with rRNA helix 68 density but no central protuberance and continuous conformational variation in the SF3b and helicase regions of the pre-catalytic spliceosome. Sample maps and links are provided in the supplementary material; code available upon publication.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Cryo-EM, heterogeneous reconstruction, gaussian splatting, differentiable rendering

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_meVzOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents CryoGaussian, a Gaussian-splatting framework for homogeneous and heterogeneous cryo-EM reconstruction with fixed input poses. It models structural variability through particle-image and per-Gaussian latent variables, followed by per-state reconstruction. The method is evaluated on synthetic benchmarks and experimental datasets.

Critical strengths
  • The framework supports both compositional and conformational heterogeneity within an explicit Gaussian representation.
  • The spatially localized Gaussian parameters offer potential interpretability advantages over implicit neural representations, with positions, scales, and amplitudes that can be directly inspected.
Critical weaknesses
  • The claimed resolution gains require stronger validation on experimental data. In Figure 3, the GSFSC remains unusually high near the Nyquist limit, while the displayed maps do not clearly demonstrate corresponding improvements in interpretable structural detail. Although the paper includes masking controls and synthetic validation, the elevated high-frequency agreement remains insufficiently explained, raising concerns about potentially inflated resolution estimates.
  • The practical advantage for heterogeneous reconstruction remains unclear. The experimental results mainly recover previously reported assembly states and conformational motions. Despite the synthetic benchmark scores, the paper does not demonstrate substantially better recovery or interpretation of heterogeneity on experimental data than established methods.
Questions for authors
  1. What explains the persistently high GSFSC near the Nyquist limit in Figures 3 and 26? Beyond half-map agreement and the synthetic validation in Figure 7, can the authors provide evidence that this high-frequency signal corresponds to accurate structural detail in the experimental reconstructions?
  2. Under matched experimental conditions, which structural states or conformational variations does CryoGaussian recover more reliably than established methods such as CryoDRGN or RECOVAR? Can this advantage be demonstrated through quantitative comparisons and specific structural examples?
1 comment
AuthorsOpenReview ↗

Response to Reviewer meVz

We thank the reviewer for their time and comments. The points raised were carefully considered in a substantial revision of the manuscript, which refocuses its scope, restructures its experimental evaluation, and strengthens the validation of results on both synthetic benchmarks and experimental data.

Reviewer_u4VvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CryoGaussian, a cryo-EM reconstruction method based on Gaussian splatting that handles both homogeneous and heterogeneous reconstruction. The density is modeled as a mixture of anisotropic 3D Gaussians. The Gaussian parameters are fit directly to the particle images through a differentiable projection model that accounts for the CTF.

For homogeneous reconstruction, the authors constrain the Gaussian widths and the number of components using quantities estimated from the data. For heterogeneous reconstruction, each particle image gets its own learnable latent code and each Gaussian gets its own learnable feature vector. The two are concatenated and fed into MLPs, which predict per-Gaussian displacements, amplitudes, and scales. Training has two stages. In the first stage, the particle latents and the Gaussian representation are optimized together. In the second stage, the latents are frozen and the density representation is refit. The authors also wrote custom CUDA code to speed up Gaussian rasterization and gradient computation.

On five synthetic CryoBench datasets, the heterogeneous results are competitive with existing methods. On experimental data, the method produces high-resolution homogeneous maps. It also recovers compositional and conformational variability that earlier studies have reported, such as ribosome assembly intermediates and motions in the pre-catalytic spliceosome.

Critical strengths
  1. The same Gaussian-based formulation is used for homogeneous and heterogeneous reconstruction, and the pieces fit together well. Because the density is represented explicitly and the projection model is differentiable, the density can be optimized directly against the particle images under a physically grounded image-formation model.

  2. The authors evaluate on all five CryoBench datasets, which include both conformational and compositional heterogeneity, and on several experimental datasets for homogeneous and heterogeneous reconstruction. This gives a reasonable picture of how the method behaves across different molecules and different kinds of structural variability.

  3. The constraints on Gaussian width and Gaussian count deal with real practical issues, namely how much capacity the representation should have and how to keep it from overfitting. The CUDA optimizations and the distributed implementation are also practically valuable, particularly for large datasets.

  4. The method recovers ribosome assembly intermediates and spliceosome conformational changes that were characterized in earlier work. These are not new biological findings, but getting known structural features back from experimental particle images is encouraging qualitative evidence that the method works in practice.

Critical weaknesses
  1. The core of CryoGaussian overlaps heavily with CryoSplat and CryoSPIRE. CryoSplat already did Gaussian splatting for homogeneous cryo-EM with random initialization and a differentiable projection model. CryoSPIRE went further and did heterogeneous reconstruction with per-particle latent codes, per-Gaussian features, and MLPs predicting Gaussian positions and amplitudes — which is essentially the same setup. The latent dimensions (4 for particles, 24 for Gaussians) are even identical to CryoSPIRE’s. What seems genuinely new is the data-driven constraints on the representation, making scales latent-dependent, the particular two-stage optimization, and CUDA-level speedups. But the paper does not isolate what these additions buy you on top of what CryoSPIRE already provides. A head-to-head comparison with CryoSPIRE and ablations targeting the actually novel pieces would go a long way toward making the novelty claim more credible.

  2. The Gaussian count rule comes from the space-bandwidth product, which estimates the effective dimensionality of a spatially localized, approximately band-limited signal. But the effective dimension of the signal space and the optimal number of Gaussian components in a nonlinear mixture are two different things. Each anisotropic Gaussian carries several continuous parameters, and the paper does not establish the mapping between signal dimensionality and Gaussian mixture capacity. The minimum FWHM constraint has a similar gap. It appeals to the Nyquist scale, but Nyquist does not say each Gaussian needs an FWHM of at least two pixels. Both constraints are sensible as heuristics; the point is that their value needs to be shown empirically through controlled experiments, not argued mainly from theory. On a related note, the Gaussian count is set from a thresholded backprojection. If the dataset has rare or low-occupancy components, those regions will be underrepresented in the support estimate. Whether a fixed Gaussian budget derived this way is sufficient for minority states is an open question.

  3. On CryoBench, CryoGaussian tops the leaderboard on three out of five datasets by FSC AUC, but the gains over the next best method are 0.001, 0.001, and 0.002. On the remaining two datasets, other methods win. That is not a strong case for a substantial advance. For homogeneous reconstruction, Figure 3 gives the same Nyquist-limited resolution for CryoGaussian and CryoSplat on all three experimental datasets. The FSC curves look different, but the final reported resolutions are identical, so it is hard to claim a real improvement. There is also a coupling issue: the FSC-based evaluation mixes reconstruction quality with heterogeneity estimation quality. It is unclear whether the method is getting better particle states, a better density representation, or just benefiting from the second-stage refit. Metrics that probe the heterogeneity inference directly, plus repeated runs with uncertainty estimates, would make the experimental story more solid.

  4. The optimization strategy is not properly justified. In the second stage, the particle latents are frozen and a brand-new Gaussian representation is trained from scratch. Why is reinitialization better than continuing from where the first stage left off? Without a comparison between single-stage training, second-stage fine-tuning, and second-stage reinitialization, it is impossible to tell whether the improvement comes from the strategy itself or just from running more optimization. The same goes for the CUDA and distributed training optimizations. They could be genuinely useful, but without end-to-end comparisons of runtime, memory usage, and scaling against the relevant baselines, the practical benefit is hard to judge.

Questions for authors

Q1. What are the two latent spaces actually doing, and how do you know? In Section 3.3, z_i and f_g are concatenated and fed into MLPs that predict the Gaussian attributes. They have different sharing patterns, z_i is per-image, f_g is shared across images, but that alone does not tell us what each one is encoding. What information is each latent space supposed to capture?

Q2. Which pieces actually matter relative to CryoSPIRE and CryoSplat? The latent-conditioned Gaussian representation is very close to what CryoSPIRE already does. Can the authors be more specific about what the methodological differences are and how much each one contributes? Controlled comparisons under matched settings and matched compute budgets would make it much easier to assess these design choices.

Q3. How robust is the heterogeneity estimation to pose errors? The method takes poses as given and learns a separate latent for each particle. If there are per-particle pose errors, those errors could leak into the latent-conditioned deformations and show up as apparent structural variability. Have the authors tested this, for instance by adding pose perturbations and seeing how the inferred conformational states change? This is especially relevant for interpreting subtle motions in experimental datasets.

Q4. Does the method actually improve heterogeneity estimation, or just density fitting? The main quantitative results are FSC-based, but a higher FSC could come from better density fitting without the heterogeneity inference itself being any more accurate. Can the authors provide metrics that evaluate the latent space directly — for example, neighborhood preservation or information-imbalance metrics for continuous variability, and classification or clustering accuracy for discrete compositional states?

1 comment
AuthorsOpenReview ↗

Response to Reviewer u4Vv

Thank you for the thorough and constructive review. The revision clarifies the roles of the two latent spaces and adds controlled ablations of the components you identified as distinct from prior work, including the data-derived constraints, latent-dependent scales, and the per-state reconstruction strategy under matched iteration budgets, together with runtime and memory comparisons. We also evaluate the inferred latents directly and report variability across training seeds. Since particle poses are inputs to CryoGaussian, we examine whether the inferred latents encode pose or CTF information. As no public implementation of CryoSPIRE is available to us, we compare against its published results, and the relationship to CryoSplat and CryoSPIRE is now discussed explicitly.

Reviewer_TBDzOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents a method for homogeneous cryo-EM reconstruction and heterogeneous reconstruction under conformational and compositional variability using a Gaussian mixture model. The method follows the conventional cryo-EM image formation model and initializes the density volume randomly with Gaussians or based on a consensus optimization. It recovers conformational and compositional variability by assigning a latent variable to each particle image and to each Gaussian. These latent variables are concatenated and mapped to Gaussian displacements and amplitudes using a multi-layer perceptron. In this way, the Gaussians can either be removed or their positions modulated to account for the variability in the particle images.

The method appears to perform competitively with previous work, but it's unclear whether it represents a significant advance. In particular, key elements of the method seem quite similar to previous techniques: in the homogeneous case, the method appears similar to other work on GMM-based cryo-EM reconstruction (e.g., cryoSplat), and the heterogeneous case appears quite similar to cryoSPIRE. I couldn't glean from the text what the key methodological differences are, and to which specific aspects of the approach I should attribute any performance differences.

Critical strengths

The methodology is mostly clear, and the work appears to perform competitively with baselines.

The method is also evaluated on real datasets and appears to demonstrate achieving some level of compositional or conformational variability (though it is always difficult to assess how accurate such reconstructions are in the absence of ground truth).

Critical weaknesses

My main concern is that I didn't understand the positioning of the work. In particular, the introduction does not clearly state the key differences of the methodological approach compared to previous techniques (what limitations of previous work are addressed here?).

The also method bears strong resemblence to other techniques. In the homogeneous case, what is the difference between this approach and previous GMM-based methods such as cryoSplat? In the heterogeneous case, the main difference with cryoSPIRE seems to be the absence of anchor Gaussians. My understanding of that method is that the anchor Gaussians were required to regularize the recovered conformational states—why are they not required here?

The evaluations on real datasets are also missing a comparison to cryoSPIRE, which seems to be the closest related work (Figs. 3, 4).

Finally, certain figures are extremely dense and lack detailed captions that would make them easier to parse — see my more detailed comments in the next section.

Questions for authors
  • Figure 1 is very busy/dense with small font and many different panels. Also, the caption does not provide any overview or explanation of the panels, which makes it harder to parse. I suggest to provide a high-level overview of the takeaways for each section (a-c), which would make it easier to understand.

  • I had a hard time following the introduction: it wasn't clear what exactly this approach is doing that is different than previous GMM representations that handle compositional or conformational variability, e.g., CryoSPIRE or DynaMight. Some text is there that describes the general difference in approach, but is the proposed approach addressing any specific limitations of previous work?

  • Why not include comparisons to CryoSPIRE in Fig. 3 or Fig. 4? Do the methods recover similar compositional variation in Fig. 4?

  • Fig. 2 is also quite difficult to parse. None of the bottom plots are discussed in the caption and the images of the reconstructions are so small and low resolution that it's difficult to appreciate the results.

  • The representation seems to closely follow CryoSPIRE (e.g., both use per-image latents and per Gaussian latents, with the latter being decoded to retrieve displacement and amplitude). However, this similarity doesn't seem to be acknowledged or discussed. The main difference seems to be the lack of anchor Gaussians in the proposed method?

  • In the homogeneous reconstruction case, what is the difference between this method and other Gaussian splatting-based cryoEM techniques (e.g., cryoSplat or DynaMight).

1 comment
AuthorsOpenReview ↗

Response to Reviewer TBDz

Thank you for the detailed comments. The revised manuscript states explicitly the limitation of prior Gaussian-based methods that CryoGaussian addresses, and how it differs from CryoSplat, DynaMight, CryoSPIRE, and related methods. This includes how our formulation differs from the anchor-based design of CryoSPIRE, while acknowledging the design elements the two share. As we were unable to find a publicly available implementation of CryoSPIRE, we compare against its published results on the benchmarks where they are reported. Figures have been redesigned, with captions that explain each panel.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.