- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CARNIVAL, a simulation-based representation learning framework for protein classification and localization in cryo-ET tomograms. The framework constructs paired views by varying imaging conditions, background, and protein orientations, and combines view consistency with protein identity supervision, distributional regularization, and auxiliary segmentation. Trained only on simulated data, the model is evaluated without finetuning on experimental Phantom and in-cell datasets. Compared with TomoTwin, it improves classification and detection on Phantom, improves classification on C. reinhardtii, and extracts feature volumes faster, but performs worse in detection on five of seven in-cell tomograms.
Critical strengths
-
Evaluation on experimental data. The authors evaluate both classification and detection on a controlled Phantom sample and two cellular systems, covering different acquisition and processing conditions. These experiments provide valuable evidence about transfer to experimental data and reveal limitations that would not be apparent from synthetic evaluation alone.
-
Detailed data-generation procedure. The scene construction, relationships between the six views, and ranges of imaging and reconstruction parameters are described in useful detail. This makes the proposed data-generation strategy understandable and facilitates its reproduction.
Critical weaknesses
-
The gains on Phantom do not establish improved generalization to the intended in situ application. A central motivation for cryo-ET is to identify macromolecules within their native cellular context, including their spatial organization and associations with membranes, filaments, organelles, and other complexes. Performance in this environment is therefore particularly important for assessing the practical value of the proposed annotation framework. However, CARNIVAL underperforms TomoTwin in detection on five of seven in-cell tomograms, despite improving classification in some settings. Classification at known particle positions does not resolve this limitation, since it bypasses the task of locating targets within the cellular background.
The Phantom benchmark provides valuable experimental validation, but it tests a different setting: purified particles are mixed with cellular lysate to create a controlled sample. Although this captures real imaging artifacts and biological background, it does not preserve the full native organization of an intact cellular environment. Its construction also shares some characteristics with the proposed simulated scenes, which place proteins among membranes and small-protein backgrounds. A plausible alternative explanation is therefore that the new training distribution is better matched to Phantom, rather than that the proposed training framework learns representations that generalize better to native cellular environments. The combination of improved Phantom detection and weaker in-cell detection makes this distinction consequential, although it does not establish the cause.
The current experiments do not adequately resolve this alternative explanation. TomoTwin uses an existing pretrained checkpoint, so both the training data and learning framework differ between methods. Retraining TomoTwin on data from the proposed simulator, while retaining its original objective, would help isolate the effect of the training data. Moreover, A1 matches the reference recipe on Phantom classification and outperforms it on in-cell classification, leaving the specific benefit of physics-based view pairing unclear. Thus, the results demonstrate useful transfer to experimental data, but do not sufficiently establish that the proposed mechanism improves generalization to the paper’s central application. Comparisons with recent related methods, such as CryoSiam and ProPicker, are also needed to assess its added value beyond TomoTwin.
-
The central training procedure is incompletely specified, and the contribution is difficult to assess. The paper’s main methodological contribution concerns how simulated views and labels are used for representation learning. Nevertheless, the exact view-consistency objective, the pairing and weighting of the six views, the segmentation loss, and the loss coefficients are not fully specified, including in the appendix. These are central components of the proposed method, rather than incidental implementation details, and their omission hinders reproduction and assessment.
The paper also does not clearly delineate the novel components relative to the adopted backbone, LeJEPA, and previous simulation-based approaches. In particular, the narrative emphasizes physically grounded view construction, whereas the strongest ablation evidence supports scene-level identity supervision. A clearer statement of the contributions, together with evidence corresponding to each contribution, is needed to establish what methodological advance the paper demonstrates.
Questions for authors
-
Could the authors retrain TomoTwin on data generated by their simulator, while retaining its original training objective, and evaluate it on both Phantom and in-cell datasets? This would help determine whether the Phantom gains primarily reflect a better-matched training distribution rather than improved transferability of the proposed learning framework.
-
Given that in situ annotation is a central motivation, what evidence establishes an advantage for the proposed framework in native cellular environments, where detection underperforms TomoTwin on five of seven tomograms? In particular, A1 matches the reference recipe on Phantom classification and performs better on in-cell classification. Could the authors provide detection results for A1 to clarify the benefit of physics-based view pairing?
-
Please specify the exact view-consistency objective, the pairing and weighting of the six views, the segmentation loss, and all loss coefficients. Please also clarify which components constitute the principal methodological contributions relative to LeJEPA and prior simulation-based approaches.
-
Could the authors provide comparisons with recent related methods, particularly CryoSiam and ProPicker under comparable conditions, or explain any specific incompatibilities that prevent such comparisons?
5 comments
Answer to Reviewer MixS (2/3)
W2 (continued): "The paper also does not clearly delineate the novel components relative to the adopted backbone, LeJEPA, and previous simulation-based approaches. In particular, the narrative emphasizes physically grounded view construction, whereas the strongest ablation evidence supports scene-level identity supervision. [...]"
The contributions are stated clearly at the end of Section 1, each with its evidence.
- The simulator, which runs on the GPU inside the training loop and provides clean and corrupted versions of each scene with the position and identity of every protein. Evidence: its data improves TomoTwin on the Phantom and on raw C. reinhardtii (Section 4.3).
- The training methodology. Relative to LeJEPA, clean and corrupted views of one simulated scene take the place of the global and local views, and a new loss, , uses the scene labels on a dense feature volume. Evidence: the ablations (Section 4.4).
- The model, compared with TomoTwin.
We agree that the evidence is strongest for scene-level identity supervision, and the revised narrative follows it. is needed in every setting in both tasks, helps on every dataset except DS-10001, and whether the forward model or generic augmentation should separate the views is left open.
Q1: "Could the authors retrain TomoTwin on data generated by their simulator, while retaining its original training objective, and evaluate it on both Phantom and in-cell datasets? [...]"
Yes, this is now Section 4.3. We trained TomoTwin with its released architecture, loss and configuration on data from our simulator: a fixed set of 800 training and 200 validation subvolumes for each of the 112 protein types, as in TomoTwin's own training set, so that only the training distribution changes. We report the epoch with the best validation , as TomoTwin's paper does.
- Phantom and raw C. reinhardtii, where CARNIVAL leads the released TomoTwin: the retrained model lies between the two. In classification it is 0.01–0.04 above the released model and 0.02–0.04 below CARNIVAL. In detection its mean is 0.03 above the released model's on each dataset, and CARNIVAL is ahead of it on all seven volumes, by 0.07–0.12.
- So your hypothesis is partly right: the simulated data accounts for part of CARNIVAL's lead, but not all of it. The remainder is not the objective alone, since the two models also differ in architecture and CARNIVAL draws new scenes throughout training.
- The gain is not specific to the Phantom: it also appears on raw C. reinhardtii, an in-cell dataset. The simulator's protein types are those of TomoTwin's own training set above 100 kDa.
- Denoised C. reinhardtii and S. pombe: the simulated data does not improve TomoTwin. Classification is within 0.01 of the released model, except on DS-10000 (0.04 lower), and detection is lower on all seven tomograms, by 0.05–0.12.
Q2: "Given that in situ annotation is a central motivation, what evidence establishes an advantage for the proposed framework in native cellular environments, where detection underperforms TomoTwin on five of seven tomograms? [...] Could the authors provide detection results for A1 to clarify the benefit of physics-based view pairing?"
There is no general advantage across all tested in-cell datasets: CARNIVAL is ahead on raw C. reinhardtii, and behind in detection on denoised C. reinhardtii and on S. pombe (see W1).
Detection for A1 (now ) is in Table 20, against two seeds of the reference at the same 10,000 steps. It is above both seeds on 7 of 10 in-cell volumes and between them on the other 3; on the Phantom it is below both on the 4 main volumes and level with them on 22 additional ones. With one run at 10,000 steps, we cannot show that physics-based pairing adds anything over generic augmentation of one simulated acquisition, or the reverse, and the paper leaves this open. What the ablations support is what both arms share: the simulator, and .
Answer to Reviewer MixS (3/3)
Q3: "Please specify the exact view-consistency objective, the pairing and weighting of the six views, the segmentation loss, and all loss coefficients. Please also clarify which components constitute the principal methodological contributions relative to LeJEPA and prior simulation-based approaches."
See answers to W2 above, and the contributions are stated at the end of Section 1 in the paper, together with their evidence.
Q4: "Could the authors provide comparisons with recent related methods, particularly CryoSiam and ProPicker under comparable conditions, or explain any specific incompatibilities that prevent such comparisons?"
We did not run these comparisons. In the time available, we gave priority to the retraining of TomoTwin.
- ProPicker: It encodes its prompts with TomoTwin and picks with its own promptable segmentation network. Our comparison with TomoTwin therefore covers its prompt representation, but not its picking network.
- CryoSiam: As far as we understand, it identifies particles through modules trained on top of its embeddings and not by matching references, so the reference-based protocol in our paper does not apply to it directly.
Answer to Reviewer MixS (1/3)
We thank the reviewer for the constructive assessment, and in particular for proposing the TomoTwin retraining experiment, which is now in the paper. We provide our answers below, with changes in the revised PDF marked in blue.
W1: "The gains on Phantom do not establish improved generalization to the intended in situ application. [...] CARNIVAL underperforms TomoTwin in detection on five of seven in-cell tomograms, despite improving classification in some settings [...]"
We agree, and the revision says so plainly (abstract, Sections 1, 4.2 and 5). After a correction of the C. reinhardtii ribosome annotations (Appendix D), TomoTwin leads detection on all seven of these in-cell volumes, not five. The new in-cell evidence is on raw reconstructions of the three C. reinhardtii tomograms, the input that TomoTwin's workflow prescribes: there CARNIVAL leads in classification (0.964 vs 0.917) and in detection on all three tomograms (by 0.10–0.17; Tables 1 and 3). We claim no general advantage across all in-cell datasets.
W1 (continued): "[...] A plausible alternative explanation is therefore that the new training distribution is better matched to Phantom, rather than that the proposed training framework learns representations that generalize better to native cellular environments. [...]"
We agree that the Phantom is not the native cellular setting, and that this alternative explanation had to be tested: we ran the TomoTwin retraining you propose, using CARNIVAL's data (Q1). We would add only that the Phantom is experimental data built as a benchmark for this annotation task, so we consider a result on it informative in its own right.
W1 (continued): "The current experiments do not adequately resolve this alternative explanation. [...] Retraining TomoTwin on data from the proposed simulator [...] would help isolate the effect of the training data. Moreover, A1 matches the reference recipe on Phantom classification and outperforms it on in-cell classification [...]. Comparisons with recent related methods, such as CryoSiam and ProPicker, are also needed [...]"
W2: "The central training procedure is incompletely specified [...]. the exact view-consistency objective, the pairing and weighting of the six views, the segmentation loss, and the loss coefficients are not fully specified [...]"
Section 3.2 and Appendix A now specify them.