acceptodds
Under review as a conference paper at ICLR 2027

CryoFATHOM: Flow-matched Amortized Tomography for Held-Out Molecules

Abstract

Single-particle cryo-electron microscopy (cryo-EM) is a key technique for estimating the 3D density of proteins and other large molecules from randomly oriented 2D projection images. Standard pipelines use many thousands of noisy images to simultaneously estimate a high-resolution molecular density and the projection operator that corresponds to each image. Emboldened by the recent progress in generative AI, we ask if one could learn a strong prior on the distribution of protein shapes to produce useful reconstructions from a tiny number of input images. To answer this question we built CryoFATHOM, a conditional flow matching model that was trained on 850 proteins from the Protein Data Bank. The model samples a molecular density conditioned on its projection images, where for simplicity we assume known orientations. Reconstruction is done by integrating the ODE of the trained conditional velocity field, with no inference-time optimization. By picking the number of Euler steps and incorporating classifier-free guidance we can trade sharpness against fidelity. We release our code and a benchmark for evaluating trainable single-particle reconstruction methods on held-out molecules. In our experiments, CryoFATHOM produced better resolution reconstructions than RELION up to about 100 projection images. Potential applications of our approach include the reconstruction of molecules with continuous heterogeneity, cryo-ET, and medical imaging methods such as CT and MRI.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Cryo-EM, conditional flow matching, generative models, tomography, 3D reconstruction, diffusion transformers

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JPLiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CryoFATHOM, a generative approach that reconstructs a 3D density map from sparse, posed multi-view cryo-EM particle images using a conditional flow-matching model. Conditioned on patch-wise features of the projection images and their orientations, the model estimates the velocity field that transports Gaussian noise toward the target density volume. The model is trained on a synthetic dataset of 850 molecules collected from the PDB after sequence-based deduplication, and is evaluated on 75 held-out molecules. Experiments show that CryoFATHOM achieves the best performance with few input images (k = 4–64), whereas the baselines outperform it as the number of images increases (k ≥ 128). Ablation studies confirm the need for orientation information and show the impact of the patch size.

Critical strengths
  1. This paper tackles a challenging task in cryo-EM, and the approach is technically sound. In particular, it proposes a novel way to combine image features and orientation features as conditions that guide the generation of the final estimated volume. On the synthetic dataset, the results are compelling when only a few input views are available.
  2. The evaluation protocol on the synthetic dataset is reasonable, with multiple training seeds and reported variance.
  3. After a careful reading, I find the pipeline reproducible.
Critical weaknesses
  1. Ideal settings may lead to no real-world applications. Though the table 1 has shown the strength to learn a geometry prior of the density map when the input view is sparse on the held-out synthetic data, there is no evidence the same pipeline can generalize to the real-world dataset. For example, the image poses of the cryo-EM is very hard to obtain given sparse inputs, the refinement of poses are essentially equivalent to the refinement of density volume, it is nearly impossible to get the perfect poses without a high-quality reconstruction. This main concern leads me to has a low expectation that this approach can make real impact at the current form. I would like to suggest the author discuss how can we further remove the need for precise poses such as making a pure ab-initio pipeline like cryoDRGN-AI or CryoFastAR, it may make a lot of sense to me. Nevertheless, the resolution of density volume achieved currently is still quite low even for ab-initio reconstruction.

  2. Hallucination is not analyzed. Structural biology depends on recovering reliable, reproducible structures from raw data, so a generative prior should be used conservatively, with theoretical and experimental analysis of whether it hallucinates and how to detect or avoid it. The paper only measures the sharpness–fidelity trade-off in aggregate (Table 8) and never discusses hallucination, uncertainty or failure detection. Figure 11 shows the risk: at k=8k=8 the classical methods fail visibly, whereas CryoFATHOM returns clean, apparently high-resolution maps that can be wrong. For 6LHT, all samplers produce a compact body with a single apex instead of the true core with peripheral protrusions, yet App. J states that the model “usually” matches the ground truth and never analyzes the exceptions. Whatever the causes (e.g., dataset or training limitations), such confident failures are hard to detect without ground truth.

Minor points:

  1. Figure 2 shows patchification before CNN feature extraction, whereas the method section (Sec. 4.2) and Table 5 describe the opposite order. Please clarify which order is used.

  2. The Introduction devotes considerable space to potential future applications, none of which are supported by experiments on real data. I suggest substantially shortening this discussion, or moving it to a brief outlook in the Conclusion, so the paper stays focused on what it actually demonstrates.

  3. The word “tomography” in the title may mislead readers. The task addressed here is single-particle analysis (SPA), or cryo-EM reconstruction more broadly, whereas “tomography” usually implies cryo-electron tomography (cryo-ET). I recommend revising the title to describe the setting accurately.

Questions for authors

Please see the weakness section, how can CryoFATHOM expand to the real-world applications with a more thorough hallucination analysis?

2 comments

We thank the reviewer for their time and careful reading.

We first frame our view of this work. To the best of our knowledge, this is the first work that learns a prior over 3D densities across many molecules and that can reconstruct molecules it has never seen without any inference-time optimization. If the reviewer agrees, we kindly ask to treat this paper as a proof of concept with significant conceptual & technical novelty (see the table at page 3) and not as a finished product, as such it certainly has limitations as the reviewer pointed out. Nevertheless, we believe this kind of work is of interest to the ICLR audience.

W1 - “Ideal settings may lead to no real-world applications. [...]”

We agree that assuming good pose estimates for sparse inputs is unrealistic in some settings, yet we would like to motivate potential applications with the following examples: (i) In continuous heterogeneity analysis the poses are often estimated from the whole dataset, which allows good estimates up to the errors caused by conformational changes, but each conformation is then reconstructed from far fewer images. (ii) Cryo-ET, where the tilt angles of each series are recorded, is another potential application. We agree that the paper should describe the potential applications better. We revised the introduction accordingly (lines 87-99). We also agree with the reviewer's suggestion that ab-initio reconstruction is an excellent direction for future work. Our method is currently positioned downstream of pose estimation. We added this direction to the future work section (lines 512-514).

We also added an analysis of robustness to pose estimation errors (Section 6.2, lines 450-457 and Figure 4; details in Appendix H). Every method (CryoFATHOM, CG and RELION) receives the same poses, perturbed by realistic errors based on reported pose accuracies, or by a pessimistic setting twice as large. All methods degrade, yet we were happy to see that CryoFATHOM is the most robust and thank the reviewer for motivating such an experiment which strengthens the paper. For example, at k = 16 realistic errors cost CryoFATHOM 0.9% of its FSC resolution, against 2.9% for CG and 3.4% for RELION.

W2 - “Hallucination is not analyzed. [...]”

Regarding hallucination analysis, we hope the following relieves your concern. While our framework relates to generative models, it is not generative in the literal sense. Like any reconstruction method in an ill-posed regime, some details must be "invented", and this is determined by the prior. Whether our learned prior is riskier than the priors other methods rely on should be evaluated with quantitative metrics, as we do using the Fourier Shell Correlation. We do not claim that our low-k reconstructions can be trusted at every frequency, and the FSC does not support that. We do claim that we capture the low frequencies of the 3D volume, which correspond to the coarse shape, much better than the classical baselines. Our method is no different from any other reconstruction method in this sense: our "hallucinations" come from the training-set distribution, while those of other methods come from their analytical priors. All the methods fail at reconstructing 6LHT at k=8 projection images. We now state in Appendix J (lines 1612-1614) that some reconstructions still miss parts of the shape, naming 6LHT.

M1 - “Figure 2 shows patchification before CNN feature extraction [...]”

Thank you for catching this. This was an error in the figure that we corrected. In the revision (Figure 2, page 5), the CNN is applied now to the whole projection image before the patchification, matching Section 4.2 and Table 5.

M2 - “The Introduction devotes [...]”

We revised the text regarding potential applications (lines 87-99). We devote a single paragraph to these future applications. By placing it in the introduction we are able to motivate the unusual sparse regime with known poses that is the focus of our paper.

M3 - “The word tomography in the title [...]”

Although our method in this paper focuses on single-particle cryo-EM the framework can be used to train models for any tomographic reconstruction task with known poses, including cryo-ET and medical imaging. Hence we believe the title presents the method accordingly.

Q1 - “how can CryoFATHOM expand to the real-world applications [...]”

Regarding expanding to real-world applications, our plan for upcoming work is to analyze real data, for example by pre-training on synthetic data followed by fine-tuning on real EMPIAR data. As for hallucination, we hope our explanation for W2 is satisfactory. For real world reconstructions where ground truth is not available the standard approach in single particle cryo-EM is to use half-map reconstructions and compute the gold standard FSC with the 0.143 threshold.

I thank authors' feedback. But I have to decide to keep my score as the quality of this paper, in my perspective, is below the bar of a solid machine learning application paper in the cryo-EM field. The main reason leading to this is the lack of the real-world impact and the poor paper writing logistics. I encourage the authors explore more meaningful applications including sparse view SPA ab-initio reconstruction and cryo-ET sub-tomo averaging in the generative and feed-forward way. For the technical paper writing, please consider focusing on your precise contribution of your work instead of talking potential applications without direct experiments.

Reviewer_Ec4xOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents CryoFATHOM, a pipeline based on a conditional flow matching model to reconstruct molecules from a few cryo-ET projections used as its conditioning signal. This pipeline is composed of a permutation-invariant encoder that extracts tokens from a variable number of tomographic projections. It also includes a DiT velocity network that reconstructs the target molecule volume by posterior sampling using the previously extracted tokens as a conditioning signal. This architecture is trained on a benchmark of 850850 molecules at a resolution of 64364^3.

The paper shows that up to 100 projections, CryoFATHOM outperforms the baselines (CG and RELION), but for larger projections, its performance is no longer better than the baselines'.

Critical strengths

The problem tackled in this paper is well-motivated, and could be extended to a bench of other applications as mentioned by the authors.

The paper is overall well-written, and the method is clearly explained.

The architecture design of cryoFATHOM handles any number of projections in any arbitrary input order. Moreover, the number of solver steps and CFG are set at the inference time. These two aspects make this approach very flexible and easy to deploy for real-world captured molecules reconstruction.

Critical weaknesses

Despite the promising results, I do not believe the work is yet ready for publication. Several important aspects are deferred to future work, most notably the evaluation of the model's sensitivity to errors in the provided orientations and its validation on real experimental data.

Usually in Cryo-ET, the Fourier shell correlation is also evaluated with the threshold of 0.143, to assess the quality of the reconstruction. A complementary analysis could be added with this threshold.

The authors used only two baselines for their comparison.

There are too many words used generally by Claude LLMs (held-out, amortizes for example) they are overused in this paper (including in the title). I suggest the authors to replace them with more common synonyms.

Questions for authors

The number of 850 assets seems to be too low for learning a prior for reconstructing random molecules. Could the authors evaluate the impact of training set size on model performance? For example, by running another training with 500 assets only, to evaluate the discrepancy between the two trained models.

Could the authors provide a discussion about possible explanations why the learned prior do not outperform baselines for a large number of projections?

Usually, for real captured data, the heating causes some deformation of the molecules. This degraded the quality of the projections. How could cryoFATHOM handle this issue?

2 comments

We thank the reviewer for their time and their helpful suggestions.

We first frame our view of this work. To the best of our knowledge, this is the first work that learns a prior over 3D densities across many molecules and that can reconstruct molecules it has never seen without any inference-time optimization. If the reviewer agrees, we kindly ask to treat this paper as a proof of concept with significant conceptual & technical novelty (see the table at page 3) and not as a finished product, as such it certainly has limitations as the reviewer pointed out. Nevertheless, we believe this kind of work is of interest to the ICLR audience.

W1 - “[...] evaluation of the model's sensitivity to errors in the provided orientations and its validation on real experimental data.”

We added an analysis of robustness to pose estimation errors (Section 6.2, lines 450-457 and Figure 4; details in Appendix H). Every method (CryoFATHOM, CG and RELION) receives the same poses, perturbed by realistic errors based on reported pose accuracies, or by a pessimistic setting twice as large. All methods degrade, yet we were happy to see that CryoFATHOM is the most robust and thank the reviewer for motivating such an experiment which strengthens the paper. For example, at k = 16 realistic errors cost CryoFATHOM 0.9% of its FSC resolution, against 2.9% for CG and 3.4% for RELION. Regarding validation on real experimental data, our method is a proof of concept and is not quite ready for real data, nor do we claim so. For example, our 3D grid (64³ at 6 Å per voxel, a 12 Å Nyquist limit) is not adequate for this. Since we believe the success of real data requires substantial work, we plan in upcoming research to improve resolution recovery, robustness to estimation errors and real noise models, which will open the way to real data experiments.

W2 - “Usually in Cryo-ET, the Fourier shell correlation [...]”

To the best of our knowledge, when a reconstruction is compared to a ground-truth map, as in our setting, the standard threshold is 0.5; 0.143 is the threshold for comparing two half-map reconstructions in the absence of a ground truth (Rosenthal & Henderson, 2003).

W3 - “The authors used only two baselines for their comparison.”

This is a justified limitation, yet our setting is a "new cell": out-of-dataset generalization without inference-time optimization (see table in page 3). As discussed in Section 2, to the best of our knowledge, existing learned methods that reconstruct the density either fit a representation to each dataset or steer an unconditional prior with inference-time optimization, so they are not an adequate baseline for our method. We therefore compare against strong analytical baselines: the industry standard RELION under three CTF treatments, and conjugate gradient given the exact forward operator and noise level (an oracle inverse). Our evaluation also covers 75 test proteins never seen in training, with multiple draws and training seeds and reported variance, which we feel strengthens our empirical results even without many baselines.

Q1 - “The number of 850 assets seems to be too low for learning a prior [...]”

We agree with the reviewer that the training set size is an important choice that can significantly affect the model performance. To understand its impact we added an analysis to the revision (Section 6.2, lines 486-491; Appendix H and Figure 12). We trained the model on random subsets of 150 to 850 molecules, for each size choosing the best training budget on the validation set. The resolution improves as the training set grows, with diminishing returns: at k = 16, going from 150 to 250 molecules gains 4.1 Å, while going from 750 to 850 gains 0.2 Å. For the suggested 500 molecules, our 450- and 550-molecule models are 4.3 Å and 2.1 Å worse than the 850-molecule model at k = 16. We read 850 as a reasonable size for this model; this does not mean the method cannot benefit from more data, since data and model size usually scale together, and we would like to scale both in future work.

Q2 - “Could the authors provide a discussion about possible explanations [...]”

This is a great question which unfortunately we do not have a definitive answer to. One possible explanation to this failure mode relates to saturation in the cross-attention maps, which prevents the network from using many projection images effectively. Since this is a conjecture about an unsolved limitation, we preferred not to add an ungrounded discussion to the paper; the limitation is stated in Section 6.1 and the conclusion.

Q3 - “Usually, for real captured data, the heating causes some deformation [...]”

The challenge of molecular deformation is relevant to all reconstruction methods in the field. Methods which rely on data-driven priors could potentially adapt to such deformations better than classical methods like RELION which rely on analytical priors. We believe this could be a future benefit of our approach.

I would like to thank the authors for the rebuttal. Unfortunetly, I still believe this work is not ready for publication. I will retain my score.

Reviewer_UgsROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors introduced cryoFATHOM, a conditional flow matching framework for cryo-EM reconstruction that learns structural priors across proteins to reconstruct from a small number of 2D projections with known poses. The method combines a geometry-aware projection encoder with a 3D DiT. Experiments on simulated data show that CryoFATHOM outperforms traditional methods like RELION when fewer than approximately 100 projections are available.

Critical strengths
  • The paper is very well written and accessible even to readers unfamiliar with the field. It also includes extensive and informative ablation studies.
  • The encoding scheme for particle images and their orientations is interesting, well designed, and appears to work effectively.
Critical weaknesses
  • The assumption of known poses significantly simplifies the problem and limits its practical relevance. The authors assume that particle orientations are provided by an upstream processing stage (L169 - 170), which is unrealistic. In practice, one of the main reasons why cryo-EM reconstruction is challenging with very few particles is the difficulty of estimating accurate poses. Pose estimation and high-resolution density reconstruction are inherently coupled, and accurate poses generally cannot be obtained without a sufficiently good reconstruction. Therefore, assuming that reliable poses are already available from upstream processing largely bypasses one of the central challenges of the reconstruction problem.

  • Lack of experiments on real cryo-EM data. Real datasets often have non-uniform orientation distributions, imperfect particle centering, and more complicated noise model rather than white noise. It is unclear how robust CryoFATHOM would be under these more realistic conditions. Testing on experimental data should be relatively straightforward using an existing EMPIAR dataset with the final particle stack and poses (e.g. EMPIAR-10028).

  • There are some reconstructions exhibit noticeable artifacts despite achieving better quantitative metrics in Figures 11-13. Even in the low-particle regime (k = 8, 16, 32) where CryoFATHOM generally outperforms the classical baselines, some reconstructed densities (e.g. 7DGE, 7ADO, 6KIU) appear to contain artifacts than those produced by conventional methods like RELION. These examples raise concerns about whether the learned structural prior may introduce artificial features that are not supported by the input particles.

Questions for authors

For the experiments (both training and inference), why not use EMDB since it's more realistic densities from cryo-EM? Why use PDB and simulate densities from the atomic models instead?

2 comments

We thank the reviewer for their time and thoughtful feedback.

We first frame our view of this work. To the best of our knowledge, this is the first work that learns a prior over 3D densities across many molecules and that can reconstruct molecules it has never seen without any inference-time optimization. If the reviewer agrees, we kindly ask to treat this paper as a proof of concept with significant conceptual & technical novelty (see the table at page 3) and not as a finished product, as such it certainly has limitations as the reviewer pointed out. Nevertheless, we believe this kind of work is of interest to the ICLR audience.

W1 - “The assumption of known poses significantly simplifies the problem and limits its practical relevance [...]”

Reconstruction from given poses has a well-established role next to ab-initio methods: standard pipelines include a dedicated reconstruction stage that takes the poses as input, and ours can naturally follow pose estimation by RELION or CryoFastAR. The setting is also highly relevant to continuous heterogeneity analysis where the poses are estimated from the whole dataset, which allows good estimates up to the errors caused by conformational changes, but each conformation is then reconstructed from far fewer images. Cryo-ET, where the tilt angles of each series are recorded, is another potential application. We agree that the paper should motivate these settings better, and we revised the introduction accordingly (lines 87-99).

We also added an analysis of robustness to pose estimation errors (Section 6.2, lines 450-457 and Figure 4; details in Appendix H). Every method (CryoFATHOM, CG and RELION) receives the same poses, perturbed by realistic errors based on reported pose accuracies, or by a pessimistic setting twice as large. All methods degrade, yet we were happy to see that CryoFATHOM is the most robust and thank the reviewer for motivating such an experiment which strengthens the paper. For example, at k = 16 realistic errors cost CryoFATHOM 0.9% of its FSC resolution, against 2.9% for CG and 3.4% for RELION.

W2 - “Lack of experiments on real cryo-EM data [...]”

Regarding experiments on a real EMPIAR dataset, our method is a proof of concept and is not quite ready for real data, nor do we claim so. For example, our 3D grid (64³ at 6 Å per voxel, a 12 Å Nyquist limit) is not adequate for this. In upcoming work we aim to improve resolution recovery, robustness to estimation errors and real noise models, which will open the way to real data experiments.

W3 - “some reconstructions exhibit noticeable artifacts [...]”.

Regarding artificial features in the low-k regime, by construction, when measurements are scarce and the problem is ill-posed, any reconstruction method has artifacts that come from its prior. Since there is no established way to compare reconstruction artifacts visually we rely on the established FSC resolution metric. As the reviewer noted, our method outperforms the baselines in this regime on this metric. Upon visual inspection, RELION and CG carry many artifacts as well, as they must in this regime, and we do not think our method is any different in that matter. If our method invented many implausible details relative to the baselines, the FSC would penalize it, which is not the case. We agree that some reconstructions still miss parts of the shape, and we now state this explicitly in Appendix J (lines 1612-1614).

Q1 - “why not use EMDB since [...]”

Regarding EMDB versus PDB, we render densities from PDB atomic models because the benchmark needs an exact ground truth for every molecule. Deposited EMDB maps are themselves reconstructions: they carry noise, non-uniform local resolution, masking and sharpening choices, and the priors of the software that produced them, so training and scoring against them would mix these effects into what counts as correct. Atomic models let us render every molecule with one controlled recipe (ChimeraX molmap at 3 Å), the same recipe CryoBench uses for its Tomotwin-100 set. In future work we will certainly tackle also EMDB data as part of the training set either in pre-training or fine tuning stages.

I would like to thank the authors for the response. I understand that this is a work of proof-of-concept, but the problem setting, especially the known-pose assumption is not realistic in the real world. If the particles already have fairly accurate poses, there must be a corresponding density with a decent resolution, and this actually defeat the purpose of the paper. I therefore decide to keep my rating.

Comments on the paper

1 comment

To the reviewers and AC we would like to note that we submitted a revision based on the initial feedback and would be happy to engage with further questions or comments on our responses while the discussion period is still in process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.