- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a meta-algorithm that orchestrates stochastic multiclass ab initio cryo-EM jobs and combines their particle assignments. Ensembles are generated by repeating jobs with different seeds and recursively classifying their output subsets. A coverage-aware weighted co-assignment matrix is sparsified and clustered with average linkage to produce final particle labels. Per-class refinement then yields reconstructed volumes. Using cryoSPARC as the base tool, the method reaches 97.4% classification accuracy on Tomotwin-45-asym and 75.1% on Tomotwin-100, and recovers four major assembly states from the unfiltered EMPIAR-10076 stack.
Critical strengths
- The method addresses a concrete workflow bottleneck. It formalizes repeated classification and subdivision without changing the underlying reconstruction software. Coverage-aware aggregation is particularly relevant when jobs operate on overlapping subsets rather than a common complete dataset.
- The synthetic results go beyond a clustering-only demonstration. On the 45-class dataset, classification and reconstruction accuracy is high, and reconstructed volume visualizations are provided. The 100-class result demonstrates a substantial increase in the complexity handled by the workflow.
- It is tested on a real-data experiment.
Critical weaknesses
- The main result is based on Hyperparameter selected on a headline 45-class evaluation. 97.4% is a tuned result on that benchmark rather than a held-out estimate of parameter-selection performance. An independent mixture benchmark or a label-free selection rule is important, particularly when the number of underlying species is unknown and the final dendrogram cut must also be chosen, which we often face in real day experiments.
- The strongest accuracy comparisons are not controlled for output capacity or compute. The best one-shot baseline uses K=16 on balanced 45- and 100-class datasets. Under the majority-vote mapping, its accuracy is capped at 16/45 and 16/100. The neural baselines also receive much smaller budgets; Hydra uses only four classes. These comparisons demonstrate the difficulty of the task, but do not isolate the benefit of the proposed aggregation. The included single-tree result is helpful.
- Rare-class recovery and abundance imbalance remain unresolved. Both synthetic datasets contain 1,000 images per class, whereas the weight heuristics explicitly penalize strongly imbalanced partitions and small clusters. Such penalties may suppress useful partitions when real populations are skewed. The real-data experiment recovers major states but misses rare A and F. Controlled abundance and junk-fraction experiments, with per-species recall and reconstruction quality, would test this limitation directly.
Questions for authors
When there are no actual classification labels and the number of true categories is unknown, how to determine the number of neighbors and the final number of categories? Can these choices be applied to independent mixtures?
Rebuttal reply to reviewer nLkV
We thank the reviewer for the constructive feedback and for recognizing that our method addresses "a concrete workflow bottleneck," as well as the practical value of coverage-aware aggregation and our results on challenging reconstruction benchmarks. We have added several experiments to address the concerns about hyperparameter selection, baseline comparisons, and rare-class recovery.
W1, Q1. Ground-truth dependence and hyperparameter selection.
We agree that selecting the number of clusters and aggregation hyperparameters without ground truth is important for practical use. We added two analyses:
These results provide evidence of robustness and a possible label-free selection criterion. We acknowledge that automatic determination of the number of classes remains unsolved; in practice, the dendrogram permits exploration of different classification resolutions, informed by the resulting reconstructions.
W2. Baseline comparisons and computational budgets.
We agree that the single-job and neural baselines are not matched for output capacity or compute, and have clarified the accuracy ceiling of the K=16 baseline in Table 1. These comparisons are intended to illustrate the limitations of existing ab initio reconstruction approaches on highly heterogeneous mixtures, rather than establish superior performance under matched computational budgets. Indeed, scaling neural reconstruction methods such as Hydra to 45 or 100 classes would require substantial engineering and computational resources.
To better isolate the contributions of our approach, we have added compute-matched comparisons between flat and recursive ensembles (Result 6, App. G.5), as well as comparisons of alternative consensus methods applied to identical reconstruction jobs (Result 4, App. G.4). These experiments clarify the roles of ensemble generation and aggregation, and demonstrate that substantial improvements can be achieved through repetition and consensus alone.
W3. Rare class recovery.
We agree that rare-class discovery remains challenging. We now report balanced accuracy and additional analyses of rare states A and F in EMPIAR-10076 (Result 7; App. G.6, Fig. 20). Further subdivision identifies an F-enriched cluster and recovers an A-enriched particle set that can be refined to 8.7 Å. We would like to reiterate that our ensembles are, to some extent, limited by the capabilities of the constituent ab initio reconstruction methods. Future work could incorporate additional reconstruction and classification jobs, together with weighting schemes better suited to highly imbalanced mixtures.