acceptodds
Under review as a conference paper at ICLR 2027

A meta-algorithm for ab initio reconstruction of complex mixtures in cryo-EM

Abstract

We describe a systematic approach for spawning and aggregating multi-class cryo-EM reconstruction jobs. This approach formalizes standard ad hoc strategies of iterative classification and filtering typically used by practitioners to sort impure, heterogeneous samples. To our knowledge, this is the first method that can successfully perform *ab initio* reconstruction on datasets containing dozens of distinct species. We obtain 97% accuracy on *ab initio* reconstruction of a 45-class subset of Tomotwin-100, 75% accuracy on the full Tomotwin-100 dataset, and demonstrate recovery of ribosomal assembly states from an unfiltered experimental cryo-EM dataset. Our approach's capability scales with compute and lays the foundation for automated cryo-EM workflows in modern experimental settings.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
cryo-EM, scene reconstruction, ab initio reconstruction, heterogeneous reconstruction, classification, clustering, ensemble learning, structural biology

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_nLkVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a meta-algorithm that orchestrates stochastic multiclass ab initio cryo-EM jobs and combines their particle assignments. Ensembles are generated by repeating jobs with different seeds and recursively classifying their output subsets. A coverage-aware weighted co-assignment matrix is sparsified and clustered with average linkage to produce final particle labels. Per-class refinement then yields reconstructed volumes. Using cryoSPARC as the base tool, the method reaches 97.4% classification accuracy on Tomotwin-45-asym and 75.1% on Tomotwin-100, and recovers four major assembly states from the unfiltered EMPIAR-10076 stack.

Critical strengths
  1. The method addresses a concrete workflow bottleneck. It formalizes repeated classification and subdivision without changing the underlying reconstruction software. Coverage-aware aggregation is particularly relevant when jobs operate on overlapping subsets rather than a common complete dataset.
  2. The synthetic results go beyond a clustering-only demonstration. On the 45-class dataset, classification and reconstruction accuracy is high, and reconstructed volume visualizations are provided. The 100-class result demonstrates a substantial increase in the complexity handled by the workflow.
  3. It is tested on a real-data experiment.
Critical weaknesses
  1. The main result is based on Hyperparameter selected on a headline 45-class evaluation. 97.4% is a tuned result on that benchmark rather than a held-out estimate of parameter-selection performance. An independent mixture benchmark or a label-free selection rule is important, particularly when the number of underlying species is unknown and the final dendrogram cut must also be chosen, which we often face in real day experiments.
  2. The strongest accuracy comparisons are not controlled for output capacity or compute. The best one-shot baseline uses K=16 on balanced 45- and 100-class datasets. Under the majority-vote mapping, its accuracy is capped at 16/45 and 16/100. The neural baselines also receive much smaller budgets; Hydra uses only four classes. These comparisons demonstrate the difficulty of the task, but do not isolate the benefit of the proposed aggregation. The included single-tree result is helpful.
  3. Rare-class recovery and abundance imbalance remain unresolved. Both synthetic datasets contain 1,000 images per class, whereas the weight heuristics explicitly penalize strongly imbalanced partitions and small clusters. Such penalties may suppress useful partitions when real populations are skewed. The real-data experiment recovers major states but misses rare A and F. Controlled abundance and junk-fraction experiments, with per-species recall and reconstruction quality, would test this limitation directly.
Questions for authors

When there are no actual classification labels and the number of true categories is unknown, how to determine the number of neighbors and the final number of categories? Can these choices be applied to independent mixtures?

1 comment
AuthorsOpenReview ↗

Rebuttal reply to reviewer nLkV

We thank the reviewer for the constructive feedback and for recognizing that our method addresses "a concrete workflow bottleneck," as well as the practical value of coverage-aware aggregation and our results on challenging reconstruction benchmarks. We have added several experiments to address the concerns about hyperparameter selection, baseline comparisons, and rare-class recovery.

W1, Q1. Ground-truth dependence and hyperparameter selection.

We agree that selecting the number of clusters and aggregation hyperparameters without ground truth is important for practical use. We added two analyses:

  • Dendrogram-cut sensitivity (Result 1; App. G.1, Fig. 17): Performance remains strong across a range of cuts around and above the true class count, suggesting that the reported results are not dependent on the exact cut.
  • Label-free sparsification selection (Result 2; App. G.2, Fig. 18): ANMI, which measures agreement between the ensemble and constituent jobs without ground-truth labels, tracks accuracy and selects n_nbr = 100 on both Tomotwin datasets.

These results provide evidence of robustness and a possible label-free selection criterion. We acknowledge that automatic determination of the number of classes remains unsolved; in practice, the dendrogram permits exploration of different classification resolutions, informed by the resulting reconstructions.

W2. Baseline comparisons and computational budgets.

We agree that the single-job and neural baselines are not matched for output capacity or compute, and have clarified the accuracy ceiling of the K=16 baseline in Table 1. These comparisons are intended to illustrate the limitations of existing ab initio reconstruction approaches on highly heterogeneous mixtures, rather than establish superior performance under matched computational budgets. Indeed, scaling neural reconstruction methods such as Hydra to 45 or 100 classes would require substantial engineering and computational resources.

To better isolate the contributions of our approach, we have added compute-matched comparisons between flat and recursive ensembles (Result 6, App. G.5), as well as comparisons of alternative consensus methods applied to identical reconstruction jobs (Result 4, App. G.4). These experiments clarify the roles of ensemble generation and aggregation, and demonstrate that substantial improvements can be achieved through repetition and consensus alone.

W3. Rare class recovery.

We agree that rare-class discovery remains challenging. We now report balanced accuracy and additional analyses of rare states A and F in EMPIAR-10076 (Result 7; App. G.6, Fig. 20). Further subdivision identifies an F-enriched cluster and recovers an A-enriched particle set that can be refined to 8.7 Å. We would like to reiterate that our ensembles are, to some extent, limited by the capabilities of the constituent ab initio reconstruction methods. Future work could incorporate additional reconstruction and classification jobs, together with weighting schemes better suited to highly imbalanced mixtures.

Reviewer_ZR14OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In classic single-particle cryo-EM, many electron-microscope projection images of a single molecule (or small number of variants of a single molecule) are combined to produce a 3D reconstruction. This paper attempts to address the challenge of cryo-EM reconstruction from micrographs that contain mixtures with dozens of species, using an ensemble meta-algorithm that aggregates multiple multi-class ab-initio reconstruction jobs. The procedure proposed by the authors is based on multiple initializations with a hierarchical exploration hierarchy that outputs a weighted similarity matrix between particles that can be used to cluster the particle images. This is claimed to achieve new SOTA performance for extremely heterogeneous mixtures.

Critical strengths

A key critical strength of the paper is the method’s empirical result, demonstrating SOTA performance on Tomotwin subsets compared to the oneshot classical baseline of CryoSPARC (capped) and various neural network-based methods that the authors compared with. If the authors will be able to address my concerns about the evaluation I believe that the empirical results demonstrate strongly that ensembling using their (or a similar) approach is can be a key method in multi-species mixture ab-initio tasks.

Another key strength is that the proposed method is tool-agnostic and hence easy to adopt for practitioners that work with different tools.

Critical weaknesses

My main concern regards the potential inflation of the key results and their robustness because they depend on ground-truth information that would not be available in a real experiment.

First, the number of output classes of the final dendrogram is cut to the ground truth number (M=45 or M=100) and no results are reported for different cut levels. Second, the aggregation hyperparameters were selected as the best cell of a grid sweep scored against ground-truth labels on Tomotwin-45-asym, the same data reported in Table 1 (the same values were then reused for Tomotwin-100). This matters most for the “n_nbr” parameter which the authors demonstrate as a crucial parameter that can vary accuracy by up to 43.4 pp across the sweep (Figure 15, appendix F) and no label-free selection procedure is given for this parameter. The EMPIAR experiment uses a different set of hyperparameters (most notably the n_nbr = 400 instead of 100) without any explanations on how they were chosen.

Due to these issues, I interpret the headline metrics as an upper bound under oracle information and find it hard to assess how the method would perform without access to such information. Showing performance across a range of cut levels and with hyperparameters fixed in advance or chosen without labels would significantly improve the confidence in the paper empirical results (assuming they remain strong under such analysis).

The experimental section only includes two cryo-EM reconstruction tasks: TOMOTWIN (synthetic data) and EMPIAR-10076 (real data). I understand that there are not many datasets with extreme levels of heterogeneity but worry that without additional datasets (even synthetic ones generated by the authors) meaningful conclusions cannot be reached. I am not sure if the authors could address this point during the rebuttal timeframe but if they do, and address the other concerns, I believe this could move my evaluation from weak reject to weak accept.

Smaller (important) issues regarding the presentation:

  • The abstract states that "this is the first method that can successfully perform ab initio reconstruction on datasets containing dozens of distinct species". However, Section 4.2 demonstrates that cryoSPARC with K=M=32 achieves very good results. I'd say that 32 counts as "dozens" so the claim in the abstract needs to be weakened.

  • Table 1 should state that the K=16 one-shot baseline is bounded at 35.6% and 16% accuracy, which it nearly reaches. The ×5 improvement is better described as exceeding the K/M ceiling of a single job in my opinion.

  • Sec. 4.5 should also report per-class recall or balanced accuracy for EMPIAR-10076 alongside the pooled 75%, given the class imbalance.

  • The forward model in Eq. (1) has C_i \cdot P_{\phi_i}, which implies that the CTF acts in real-space by elementwise multiplication but it only acts like that in Fourier space. In real space it acts by convolution and the notation should reflect that (e.g. using a star operator). The noise \eta_i is written as N(0,\sigma^2). This should N(0,\sigma^2) but actually it would be better to state that it is typically modeled as colored Gaussian noise (not white noise).

Questions for authors
  1. How does performance change when the dendrogram is cut at values other than the true M (e.g. 25–60 for Tomotwin-45-asym)? Please report ARI/AMI alongside accuracy, since majority-vote accuracy does not penalize over-splitting.

  2. From Fig. 12, the rare classes A and F receive no particles (0% recall), and balanced accuracy over the seven published classes is roughly 55%, compared with the reported 75% pooled accuracy. The authors note the dendrogram was cut at 7 clusters in the hope of capturing these rare states. Do classes A and F emerge at higher cut levels (e.g., 10–15 clusters), or are they already absorbed into other classes at the level of the individual jobs?

  3. How was the EMPIAR hyperparameter set chosen, specifically the n_nbr value, and if no label-free procedure for choosing n_nbr exists how would you suggest potential users of the method should choose this value?

  4. It remains unclear to me whether the weighting procedure of the calculation of matrix S is meaningful or not. Can you please provide an ablation experiment demonstrating the results with uniform weighting? (i.e. w = 1 for all jobs), as well as with each weight factor removed individually? Since this only requires reclustering the existing ensembles, it should not require additional GPU time.

  5. The paper says typical signal-to-noise ratios are between 10^-1 and 10^-2. You should clarify what type of SNR you mean here. e.g. per particle window with windows sized to particle, or whole micrograph.

1 comment
AuthorsOpenReview ↗

Rebuttal reply to reviewer ZR14

We thank the reviewer for the thoughtful and constructive feedback, and for recognizing the strong empirical results and the practical value of our tool-agnostic approach, which is "easy to adopt for practitioners." We appreciate the specific suggestions for strengthening the evaluation, and have added several analyses addressing robustness, hyperparameter selection, and rare-class recovery.

Q1. Number of output clusters.

We now report accuracy and AMI across different dendrogram cuts (Result 1; App. G.1, Fig. 17). On Tomotwin-45-asym, accuracy remains approximately 97.5% for cuts at or above 45, while AMI peaks at the true class count. On Tomotwin-100, AMI remains relatively stable across cuts from 90 to 200. These findings show that our results are not dependent on choosing the exact ground-truth number of clusters, although automatic determination of the class count remains an open problem.

Q2. Rare states A and F.

We have expanded the EMPIAR-10076 analysis (Result 7; App. G.6, Fig. 20). At 11 clusters, an F-enriched cluster emerges, containing 3,016 particles with 40% F purity. For state A, approximately 70% of its particles remain grouped together even at 20 clusters. Two additional (K=2) jobs on this subset yield an A-enriched set with 90.6% purity, which refines to 8.7 Å in two of three seeds. While these results do not establish complete rare-class recovery, they demonstrate that further subdivision can recover states missed by the initial clustering.

Q3. Hyperparameter selection.

We investigated ANMI as a label-free criterion for selecting n_nbr (Result 2; App. G.2, Fig. 18). On both Tomotwin datasets, ANMI tracks accuracy and peaks at n_nbr = 100, the value used in our reported results. We propose this as a practical selection heuristic, while acknowledging that its generalization to independent datasets requires further validation. We also clarify that the EMPIAR-10076 settings were taken from the initial configuration of our Tomotwin sweep and scaling experiments, without using EMPIAR labels.

Q4. Weighting ablations.

We have added uniform-weight and component-wise ablations (Result 3; App. G.3). Weighting improves accuracy by 3.3 and 8.7 percentage points on Tomotwin-45-asym and Tomotwin-100, respectively. The sink factor contributes most strongly, while the remaining factors have smaller, dataset-dependent effects.

Q5. Presentation.

We have revised the SNR description and addressed the forward-model notation in the manuscript. Thank you for these comments!

Reviewer_5GXSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper deals with reconstruction tasks for cryo-EM datasets containing different species. The paper treats a cryoSPARC run (job in the original paper) as a weak clusterer. It generates jobs in batch by random repetition and hierarchical exploration, obtains the hard labels of all jobs, and aggregates them into a covering mask and job-weighted co-assignment matrix. Its job weight depends on the class ratio. Finally, the final classification can be obtained by sparsifying the matrix and then average link clustering

This paper evaluated the proposed method using the crybench Tomotwin-45-asym and Tomotwin-100. Ensembles of 257/234 jobs reach 97.4% / 75.1% accuracy, versus 34.5% / 15.2% for the best single job. On Tomotwin-45-asym the median pose error drops from 74.1° to 1.71°, while cryoDRGN2, cryoDRGN-AI and Hydra reach 4–13% accuracy with near-random poses. On the unfiltered EMPIAR-10076 stack ,the proposed method recovers and refines four main assembly states, and its merged sub-states are 75% consistent with the public labels.

Critical strengths
  1. In terms of experimental results, this method makes a huge progress on the currently unsolved benchmarks. The original CryoBench article explicitly states that the existing ab initio methods cannot recover the 100 structures of Tomotwin-100. Now, the method improves the accuracy from 34.5% to 97.4% on Tomotwin-45-asym and from 15.2% to 75.1% on Tomotwin-100, which achieves great progress.

  2. From the perspective of experimental design , its design is simple, can be carried out in parallel, and its behavior is measured and described rather than assumed.

  • Its design is simple and can be executed in parallel. Without modifying the underlying software, operations at the same layer are also independent of each other. Therefore, the waiting time will only increase with the growth of the tree's level, rather than the number of operations.

  • In this paper, 49+64 single jobs are used to sweep the K*M grid, and the actual data is used to illustrate the role and limitations of the single job, which provides a practical basis for the design of "multiple small K jobs plus aggregation"

  • The scaling study shows that the highest profit of a single tree is not more than 54.6%, and only after adding repetition can it reach 97.4%. Therefore, the profit obtained in this paper mainly focuses on "repetition + aggregation" rather than recursive subdivision itself

Critical weaknesses
  1. This method does not clearly distinguish itself from existing methods, and the core statement is somewhat exaggerated.
  • Existing jobs. When w ≡ 1, Equation 4 exactly matches the consensus matrix (the count of common clustering divided by the count of common sampling) proposed by Monti et al. (Machine Learning, 2003). However, neither Monti et al. nor Topchy's paper (tami 2005, which precisely discusses "combining weak partitions into strong clusters") is cited in the text.

  • Baseline missing.The truly novel aspects of the paper are the three working weights and the kNN sparsification, but these have never been compared with conventional methods. The entire paper neither contains an experiment where w ≡ 1, nor runs the CSPA/MCLA or graph partitioning methods that the paper itself references on the same job pool.

  • The statement is exaggerated. The abstract claims the “first method ... on datasets containing dozens of distinct species” (similarly Sec. 2.2). Sec. 4.3 claims an “order-of-magnitude expansion”, measured against Hydra’s 3-class subset. Both conflict with Fig. 3, where one default job with K = M is near-perfect up to M = 32.

  1. The evaluation relies on the true values, and the available real data evidence is limited.
  • The category number is derived from the truth value. EMPIAR chose 7. The author explains that this was done to ensure that the rare classes could also be captured. For Tomotwin, the baseline k-means also used 45 centers. Although this does not affect the fairness of the comparison, there is doubt about whether the claim of "automation, without the need for manual intervention" can be fulfilled with these figures in real scenarios.

  • No uncertainty is reported. There is no variance, which is a fact. In the scaling experiments in the paper, the differences among different strategies, "K ≥ 32 does not converge", "third-layer variation" - all these require variance.

  • The available data evidence is limited. As the only real data experiment in this paper (EMPIPAR-10076), it can only prove that this method can classify a few major classes without supervision on an unfiltered real data set. However, two rare categories were not divided at all, junk was not divided cleanly, and 75% of the experimental results had no reference frame. Therefore, this experiment cannot fully support the overall positioning of junk in Section 4.5 and "suitable for modern complex samples" in the full paper

Questions for authors

See weakness

1 comment
AuthorsOpenReview ↗

Rebuttal reply to reviewer 5GXS

We thank the reviewer for the detailed and constructive feedback, and for recognizing the "huge progress on the currently unsolved benchmarks" and the simplicity and parallelizability of our approach. We appreciate the suggestions for clarifying the methodological contribution and strengthening the evaluation, and have added new baselines, ablations, and robustness analyses to address these concerns.

W1a: Relationship to prior work. We agree that co-association and ensemble clustering are well-established techniques. We have added citations to Monti et al. (2003) and Topchy et al. (2005) and clarified that our novelty is not in co-association itself. Rather, we formalize stochastic ab initio reconstruction jobs as partial weak partitions, systematically generate them through repetition and recursive subdivision, and aggregate their results into a unified reconstruction workflow. This framework enables us to recover complex mixtures that individual reconstruction jobs fail to resolve.

W1b: Aggregation baselines and ablations. We added three sets of experiments directly addressing this concern:

  • Weighting ablations (Result 3): Compared with uniform weights, our weighting improves accuracy by 3.3 and 8.7 percentage points on Tomotwin-45-asym and Tomotwin-100.
  • Alternative consensus methods (Result 4): We compare HBGF, LWEA, MCLA, and standard co-association on the same reconstruction jobs. Multiple methods substantially outperform individual jobs, confirming that ensemble generation is a major source of the performance improvement. Our aggregation achieves the highest accuracy on both synthetic benchmarks, while HBGF performs slightly better on the experimental dataset.
  • Sparsification (Result 4): Sparsifying coverage-aware co-association improves accuracy by 52.7 and 48.0 percentage points on the two synthetic benchmarks, demonstrating its importance to the aggregation procedure.

Together, these experiments distinguish the benefits of generating an ensemble of reconstruction jobs from the additional contributions of weighting and sparsification.

W1c: Scope of claims. We have narrowed the claims in the abstract, Sec. 2.2 and Sec. 4.3 to avoid implying that earlier methods cannot reconstruct any mixture containing dozens of classes.

W2a: Choice of category number. We now report accuracy and AMI across different dendrogram cuts (Result 1; App. G.1, Fig. 17). Accuracy is stable above 45 clusters on Tomotwin-45-asym, while AMI peaks at the true class count. On Tomotwin-100, AMI is relatively stable across a broad range of cuts. This demonstrates robustness to the exact cut, although choosing the number of classes without labels remains a limitation.

W2b: Uncertainty. We now report variability across random subsamples of the constituent trees (Result 5; App. G.5, Fig. 19), including accuracy of 94.1 ± 1.7% and 71.2 ± 2.3% when using 80% of the full ensembles on the two Tomotwin datasets.

W2c: Experimental validation and rare classes. We agree that EMPIAR-10076 demonstrates recovery of major assembly states but does not establish comprehensive rare-class or junk-particle separation. We now report balanced accuracy (54.7% at seven clusters), as well as additional experiments showing partial recovery of states A and F through finer clustering and further subdivision (Result 7; App. G.6, Fig. 20). In particular, an A-enriched subset can be isolated at 90.6% purity and refined to 8.7 Å.

Reviewer_3GuxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper combines repeated cryoSPARC ab initio runs to reconstruct complex mixtures. The method reruns classification on particle subsets and uses how often particles are grouped together across runs to build the final partition.

Critical strengths

The study focuses on an important problem in single-particle cryo-EM. Closely related structures can be difficult to separate, and averaging their particles together can blur structural features and limit local resolution.

The method achieves 97.4% classification accuracy and a median pose error of 1.71° on the 45-species benchmark. It also recovers four major ribosomal assembly states at 3.4–4.0 Å after refinement from the full experimental particle stack.

Critical weaknesses

The methodological novelty is limited. Both co-assignment voting and recursive cryo-EM classification are widely used methods, it seems this study just tries to combine them for ab initio reconstruction.

The evaluation leaves the gain over simpler workflows unclear. The single-run cryoSPARC baseline outputs only 16 classes on the 45- and 100-species mixtures, and the flat consensus ensemble is tested with at most five repeats. A comparison under the same setting would help isolate the benefit of recursive splitting.

Questions for authors

How does the method compare with a standard recursive classification workflow using the same ab initio engine?

How well does flat consensus clustering perform under the same settings? On the same set of jobs, how much does weighting improve over equal-weight aggregation?

1 comment
AuthorsOpenReview ↗

Rebuttal reply to reviewer 3Gux

We thank the reviewer for the thoughtful feedback and for recognizing that our work addresses "an important problem in single-particle cryo-EM," as well as the strong reconstruction results on both synthetic and experimental datasets. We have added new experiments to clarify the methodological contribution and disentangle the benefits of ensembling, recursive subdivision, and weighting.

W1: Methodological novelty. We agree that repeated ab initio reconstruction, recursive classification, and consensus clustering are established ideas. Our contribution is to formalize these practices into a unified framework that automatically generates ensembles of reconstruction jobs through stochastic repetition and recursive subdivision, and aggregates their assignments across overlapping particle subsets. As the results demonstrate, this enables reconstruction of substantially more complex mixtures than individual jobs can resolve.

We have also clarified the relationship to previous consensus clustering methods, including Monti et al. (2003) and Topchy et al. (2005).

W2, Q1: Comparison with standard recursive classification. A single recursive tree corresponds closely to the hierarchical classification workflow commonly performed by practitioners. Our existing scaling analysis (Sec. 4.4; App. D) shows that a single tree reaches at most 54.6% accuracy on Tomotwin-45-asym, compared with 97.4% for the full ensemble. This demonstrates that the principal improvement comes from aggregating information across multiple reconstruction jobs, rather than recursion alone.

W2, Q2: Flat consensus at comparable compute. We conducted 25 additional whole-stack K=16 jobs to compare flat ensembles with recursive ensembles at similar computational costs (Result 6; App. G.5, Fig. 19). We find that flat ensembles are competitive with recursive ensembles over the evaluated range, reaching 73.2% accuracy on Tomotwin-45-asym at 64 GPU-hours.

We therefore agree that recursive subdivision is not always necessary to achieve strong results. Importantly, flat consensus is a special case of our framework, corresponding to stochastic repetition without recursive subdivision. Our framework supports both strategies and allows them to be combined when needed.

Q2: Contribution of weighting. We also added ablations on the same job ensembles (Result 3; App. G.3). Compared with equal weighting, our weighting scheme improves accuracy from 94.1% to 97.4% on Tomotwin-45-asym and from 66.3% to 75.1% on Tomotwin-100. Thus, weighting provides additional improvements, while ensemble generation and aggregation account for the principal gains.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General rebuttal reply and new findings (1/2)

We thank all four reviewers for their careful and constructive reviews. This comment gathers the revisions and new results. The revised PDF marks new text in blue, and the new analyses are presented in Appendix G.

Main contribution. We agree with the reviewers that consensus clustering is well established in machine learning, and that repeated and recursive classification are common practices in cryo-EM. Our contribution is to formalize and combine these ideas into a systematic, automated framework for ab initio reconstruction of complex cryo-EM mixtures. The key insight is that stochastic ab initio reconstruction jobs can be treated as weak clusterers, and that systematically generating and aggregating these jobs can resolve complex mixtures beyond the capabilities of individual reconstruction methods.

Result 1. Sensitivity to the number of output clusters (App. G.1). We evaluated different cuts of the same clustering dendrogram, reporting both majority-vote accuracy and adjusted mutual information (AMI), which penalizes over-splitting.

Tomotwin-45-asym25354045506090
accuracy55.076.887.497.497.597.597.5
AMI0.8080.9030.9370.9570.9560.9560.949
Tomotwin-100507090100120150200
accuracy48.764.172.775.178.979.180.1
AMI0.7280.7700.7820.7840.7890.7880.787

On Tomotwin-45-asym, accuracy remains approximately constant for cuts at or above the true number of classes, while AMI peaks at 45. On Tomotwin-100, accuracy improves gradually with additional clusters, while AMI remains relatively stable around the true class count and peaks at 120. These results demonstrate that the headline performance is not restricted to an exact dendrogram cut, although choosing the number of classes without ground-truth labels remains an open challenge.

Result 2. Label-free selection of the sparsification threshold (App. G.2). We propose using the ANMI measure of Strehl and Ghosh (2002) to find the best n_nbr sparsification threshold without using our knowledge of the dataset. ANMI measures agreement between the final ensemble partition and its constituent job partitions, without requiring ground-truth labels.

On both Tomotwin sets, the label-free ANMI agreement score follows the ensemble accuracy across different numbers of neighbors kept, and the best ANMI is reached at n_nbr = 100, the value the paper used:

Tomotwin-45-asym2550100200400700100020004000
ANMI0.42320.42350.42420.42140.42230.40390.39290.35510.3107
accuracy (%)97.195.497.494.996.892.387.265.454.1
Tomotwin-1002550100200400700100020004000
ANMI0.46290.46810.47050.46560.45900.43770.42520.35090.3245
accuracy (%)72.375.475.170.966.759.755.035.328.9
AuthorsOpenReview ↗

General rebuttal reply and new findings (2/2)

Result 3. Contribution of job weighting (App. G.3). We performed an ablation of uniform weighting and each individual weight component, holding the ensemble and downstream clustering procedure fixed.

publishedequal weightsno sinkno Keffno tiny
Tomotwin-45-asym97.494.193.495.295.2
Tomotwin-10075.166.365.774.575.1

Compared with uniform weighting, our weighting scheme improves accuracy by 3.3 percentage points on Tomotwin-45-asym and 8.7 points on Tomotwin-100. The sink factor has the largest effect, while the contributions of the remaining factors vary by dataset. Thus, weighting provides an additional benefit beyond ensembling, although ensembling itself accounts for most of the performance improvement.

Result 4. Alternative consensus methods on the same jobs (App. G.4). We experimented with different consensus methods. Below, we report the accuracy (%) and AMI.

Tomotwin-45-asymTomotwin-100EMPIAR-10076
ours97.4, 0.95775.1, 0.78474.6, 0.457
HBGF (Fern and Brodley 2004)93.5, 0.95273.2, 0.77477.7, 0.529
LWEA (Huang et al. 2018), sparsified97.2, 0.95567.3, 0.732—
co-association with coverage (Monti), sparsified94.1, 0.93366.3, 0.719—
MCLA (Strehl and Ghosh 2002)75.6, 0.79754.0, 0.59971.1, 0.462
co-association with coverage (Monti)41.4, 0.72718.3, 0.504—
the best single job34.5, 0.49015.2, 0.386—

Every consensus method outperforms the best individual reconstruction job. Our sparsification adds around 50 percentage points of accuracy to co-association based methods. HBGF, which needs no sparsification, is notably competitive. Some old methods, since they use dense matrices, are not applicable to the 50S (EMPIAR-10076) dataset due to memory constraints.

Result 5. Variability across ensemble subsamples (App. G.5). To characterize variability, we evaluated ten random subsamples containing 80% of the trees in each main ensemble. These subsamples achieve 94.1 ± 1.7% accuracy on Tomotwin-45-asym and 71.2 ± 2.3% on Tomotwin-100. We have added these statistics to the Table 1 caption and repeated the scaling analysis using up to five random draws of constituent trees per ensemble configuration (Fig. 19).

Result 6. Flat versus recursive ensembles at matched compute (App. G.5). We conducted 25 additional whole-stack K=16 jobs on Tomotwin-45-asym to measure how using only flat ensembles compares to the recursive tree formation at similar compute levels. We note that for Tomotwin-45-asym, under the given weight factors and job types, flat ensembles do as well as trees in the utility-cost frontier up to 64 GPU-hours, as far as we ran flat ensembles (Figure 19).

GPU-hoursflat: K=16 jobs, mean (range)trees nearest in cost
10.75 jobs: 57.6 (53.2–60.0)one K=8 two-level tree, 9.6 h: 52.7 ± 2.1
21.310 jobs: 65.2 (61.6–69.5)two K=8 trees, 19.2 h: 46.7 ± 4.3
42.720 jobs: 70.6 (69.4–73.6)four K=8 trees, 38.4 h: 65.4 ± 0.3
64.030 jobs: 73.2two K=16 trees, 72.5 h: 74.0 ± 2.5

Again, we note that flat ensembles (and other simpler workflows) are just a special case of the framework we are introducing. Flat consensus is simply the stochastic repetition dimension of job generation. Similarly, equal weights are also a special case of our framework, as any weighting can be applied at will. When simpler approaches do not work, our framework composes them and scales them up to increase the chance of reconstruction success.

Result 7. EMPIAR-10076: settings and per class accuracies (App. G.6).

Hyperparameter choice. (σ, β, γ, p_min, n_nbr) = (0.45, 5, 10, 1/450, 400): we simply used the starting point of our hyperparameter sweep on Tomotwin-45-asym and the scaling study's settings. No 50S dataset information was used to choose them.

Per class performance. At the cut of 7 clusters, the balanced accuracy is 54.7% against 74.6% pooled (reported) accuracy, since the rare states A and F have no cluster. When we cut at 11 clusters, F gets one (3,016 particles, with 40% being F particles). A stays in one cluster with other particles up to 20 clusters (70% of all A particles stay in this cluster), and sending two K=2 jobs on it extracts A at 90.6% purity without labels, and it refines to 8.7 Å (in two of three seeds). App. G.6 (Figure 20) shows the A map. Junk gathers into clusters of its own.

These results demonstrate partial recovery of rare states through additional subdivision, while highlighting the remaining challenges of severe population imbalance and rare-class detection.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.