- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper extends X-Cell with a staged procedure for predicting perturbation-induced expression profiles. Each inner stage ranks unresolved genes, commits selected predictions and applies lookahead supervision; optional outer passes reuse scorer-latent state. The method improves several endpoint metrics over its parent model across four Replogle-Nadig contexts and five training seeds. Biological analyses associate the learned ranks with pathways and protein complexes and compare RNA and antibody-derived tag (ADT) ranks. Ablations attribute the strongest gains to global ranking and lookahead, while a limited compute sweep favors additional inner stages over outer recurrence. The learned order describes model computation rather than measured biological time.
Critical strengths
-
The consistent improvement over the parent X-Cell model is the strongest result. With shared pretrained weights and inherited objectives, Looped X-Cell reduces mean MAE by approximately 7.6–10.4% across the four contexts, with results reported over five training seeds. Pearson Δ, DE direction and DE LFC also improve in each context. This provides useful evidence for staged prediction in the evaluated setting on a useful, broad sweep of metrics.
-
The model exposes internal ranks and stages that can be inspected and tested. The biological probes and recurrence controls are useful attempts to characterize these representations. In particular, the HepG2 experiment in Table A3 shows that scorer-latent recurrence preserves accuracy over additional passes more reliably than direct value feedback. This is a useful engineering observation.
Critical weaknesses
- The benchmark and ablations do not establish the predictive contribution of the complete architecture for transcriptomic perturbation prediction.
The improvement over X-Cell is promising, but the benchmark needs stronger simple controls and contemporary comparisons. Table 1 includes a linear baseline whose specification is insufficient to judge its strength. Recent benchmarking work [1–3] motivates including control/no-change and training-perturbation-mean predictions, alongside comparisons with STATE, STACK, the public-data version of TxPert, and LPM in the settings they support [6–9]. Licensing restrictions may legitimately prevent some comparisons. Have necessary permissions be requested, and what specific barriers remain? More broadly, this reviewer encourages developers including Xaira and Arc to enable reproducible head-to-head evaluations and share evaluation code, splits, predictions, and results. TxPert’s public-data implementation remains subject to non-commercial terms, whereas LPM’s Apache-2.0 implementation provides Replogle training configurations. For each omitted model, please identify the restriction or task mismatch and limit comparative claims accordingly.
Metric calibration is also needed to determine whether the improvements reflect perturbation-specific prediction gains rather than changes shared across perturbations. Systema [4] and Miller, Mejia et al. [5] motivate this distinction and the use of informative positive and negative controls. The reported centroid accuracy deserves credit, but I would also like calibration using an appropriate split-half or technical-replicate positive reference, uninformative negative controls, and an explicit definition of Pearson Δ, including how predicted and observed expression changes are calculated and which control expression profile is subtracted (the one in Appendix A6 erroneously copies the MAE definition). The generalization protocol should distinguish unseen perturbation identities from unseen identity–context pairs, establish whether test identities are excluded across all fine-tuning contexts, and quantify overlap with the pretraining data. The paper should state exactly which data were used for training, validation, and testing; which pretrained models they started from and what those models had already seen; and how much training and tuning each method received. This is needed to judge whether the comparisons are fair.
The ablations raise a more direct concern about the full architecture. In Table 2, global ranking plus lookahead has better reported means than the complete model on all six metrics, including MAE of 0.0611 versus 0.0618. The three-seed versus five-seed comparison does not establish superiority, but it provides no positive evidence that the additional components improve endpoint prediction. Please compare the simpler and complete variants under matched seeds and budgets, including fixed/random staged ordering, and establish the executor’s contribution or present it as an implementation choice. Likewise, the limited compute sweep and additional-pass stability experiments do not establish a general compute-scaling benefit; claims should distinguish stage-count sensitivity, recurrence stability, and improved accuracy at matched compute.
- The biological analyses do not establish the added value of the learned ordering, and the framing exceeds the demonstrated capability.
Recovering pathway or complex annotations without using them as explicit labels does not establish independence from biological knowledge, especially because the model uses external priors. CORUM rank permutations show sensitivity to gene correspondence, but do not establish an advantage over predicted expression/deltas or inherited/prior-only representations. Table A4 provides control-expression comparisons for selected Reactome terms only. The paper should check whether the learned gene ordering reveals more biological structure than the predicted expression levels, predicted expression changes, or information already present in the original model from the prior knowledge it was fed. They should use the same tests for each comparison and state what external biological information each model was given. Otherwise, the results support annotation-associated representations rather than independent biological discovery.
The RNA/ADT analysis also warrants narrower interpretation. The GO analysis should compare the selected genes against all genes whose proteins were measured and matched to RNA. Instead, it uses a broader background, which may make features of the antibody panel appear to be findings about the learned ordering. The Gene Ontology Consortium recommends using only genes that could have entered the selected list as the background (https://geneontology.org/docs/go-enrichment-analysis/). The paper acknowledges this limitation in B.3, but the main text’s claim of greater biological coherence remains stronger than the analysis supports. The protein-earlier and protein-later sets contain 6 and 56 genes, respectively, so differences in enrichment may partly reflect list size and statistical power. The weak abundance–rank-gap association does not exclude abundance, normalization, or initialization effects. The permutation statistic and dependence assumptions also require clarification.
Finally, the demonstrated endpoint prediction capability covers only a limited subset of the virtual cell vision described by Bunne et al. [10]. The paper should present its contribution as progress toward that vision. Applying the virtual cell label to a narrowly evaluated perturbation prediction model and a comprehensive model of cellular behavior obscures differences in scope and makes progress harder to assess. Similarly, the explicit acknowledgement that computational ranks establish neither biological time nor causal response paths should consistently govern the motivation and interpretation of the results throughout the paper. Although the paper states this in the limitations, the motivation and interpretation elsewhere sometimes imply a progression of biological processes that the analysis does not establish.
To make these results reproducible, please release the training and evaluation code, specify the data and pretrained models used in each experiment, and describe how the biological analyses were performed, including the genes used as the enrichment background.
References
[1] Ahlmann-Eltze, Huber, and Anders. Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines. Nature Methods (2025). https://doi.org/10.1038/s41592-025-02772-6
[2] Wenteler et al. PertEval-scFM: Benchmarking Single-Cell Foundation Models for Perturbation Effect Prediction. ICML (2025). https://proceedings.mlr.press/v267/wenteler25a.html
[3] Wu et al. PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis. NeurIPS (2025). https://proceedings.neurips.cc/paper_files/paper/2025/hash/8aee537279a66ced96319dfca3c00002-Abstract-Datasets_and_Benchmarks_Track.html
[4] Viñas Torné et al. Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variation. Nature Biotechnology (2025). https://doi.org/10.1038/s41587-025-02777-8. Implementation: https://github.com/mlbio-epfl/systema
[5] Miller, Mejia, et al. Deep Learning-Based Genetic Perturbation Models Do Outperform Uninformative Baselines on Well-Calibrated Metrics. bioRxiv (2025). https://doi.org/10.1101/2025.10.20.683304. Implementation: https://github.com/shiftbioscience/Perturbation-Models-Outperform-Baselines
[6] Wenkel et al. TxPert: using multiple knowledge graphs for prediction of transcriptomic perturbation effects. Nature Biotechnology (2026). https://doi.org/10.1038/s41587-026-03113-4. Public-data implementation: https://github.com/valence-labs/TxPert
[7] Adduri et al. Predicting cellular responses to perturbation across diverse contexts with State. Cell (2026). https://doi.org/10.1016/j.cell.2026.07.052. Implementation: https://github.com/ArcInstitute/state
[8] Dong et al. Stack: In-context learning of single-cell biology. bioRxiv (2026). https://doi.org/10.64898/2026.01.09.698608. Implementation: https://github.com/ArcInstitute/stack
[9] Miladinovic et al. In silico biological discovery with large perturbation models. Nature Computational Science 5, 1029–1040 (2025). https://doi.org/10.1038/s43588-025-00870-1
[10] Bunne et al. How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunities. Cell (2024). Public author preprint: https://arxiv.org/abs/2409.11654
Questions for authors
-
Please provide the missing contemporary and strong simple baseline comparisons, together with metric calibration against positive/negative controls? For any omitted model, what exact version, restriction or task mismatch prevents a valid comparison? These results would determine whether the reported gains represent a meaningful advance beyond just the parent model.
-
Are each context's test perturbation identities excluded from every fine-tuning context, and what overlap remains with the actual pretraining checkpoint? Please provide the split/checkpoint information needed to distinguish identity holdout from identity–context-pair holdout.
-
What evidence favors the full architecture over global ranking plus lookahead under matched seeds and budgets, and how does it compare with fixed/random staged schedules? Please distinguish stability under additional inference passes from improvements in prediction accuracy.
-
Do learned ranks improve biological recovery over predicted expression/deltas and inherited/prior-only representations under the same analysis? If those comparisons are unavailable, which narrower biological conclusions does the paper intend to retain?
-
How robust are the RNA/ADT enrichment results to using the eligible measured-and-matched protein panel as background and accounting for the unequal gene-set sizes? Please also specify the statistic used in the rank-gap permutation test and how the shuffling preserves the relevant dependence structure.
5 comments
Q2. Which perturbations were held out?
Table A1 now lists the fine-tuning and evaluation data for each setting. In Replogle–Nadig, every test perturbation was held out in the evaluated context and appeared during fine-tuning in at least one other context. Table 1 therefore measures transfer to held-out perturbation–context pairs. Appendix B.2 identifies the processed data release and the notebook defining the HepG2 split. Appendix B.5 describes the checkpoint shared by X-Cell and Looped X-Cell.
Q3. What does the full architecture add?
We agree that the simpler model is a strong comparison. Table 2 now uses the same three HepG2 seeds throughout. With four stages and lookahead disabled, all four orders reduced mean MAE relative to fine-tuned X-Cell (0.0668). The means were 0.0632 for learned order, 0.0648 for random order, 0.0645 for control mean, and 0.0627 for control variance. Learned order gave the highest mean Pearson and Overlap@N among the four. Adding lookahead to learned global ranking reduced MAE to 0.0611, compared with 0.0616 for the full model. We therefore attribute the main HepG2 MAE gains to staged prediction and lookahead.
Local ranking gives us a within-stage order we can inspect. In Marson, it recovered 34 CORUM modules and 38 same-complex target pairs missed by the other two rollout views (Figure A5). The scorer-latent model retained an MAE of 0.0609 to four decimal places at outer passes (Table A5). This shows stability across additional passes. Figure 4 varies the stage and pass counts without matching total compute.
Q4. What do the ranks recover beyond expression and the supplied priors?
In Marson, we applied the same clustering and CORUM scoring procedure to each representation. Stage assignments, global ranks, and local ranks recovered 135, 167, and 133 modules, respectively, compared with 82 for predicted expression and 71 for measured expression. Combining the three rollout views recovered 273 modules, and adding the two expression views brought the total to 413 (Figure A6).
We also repeated CORUM scoring on HepG2 after removing the five external prior tokens at inference and retaining the learned perturbation-gene embedding. Recovery decreased, although each rollout view still recovered about three times as many modules and five to six times as many same-complex pairs as its shuffled control (Table A8). The checkpoint was trained with the priors, so this experiment measures the effect of supplying them at inference. For Reactome, the learned stages recovered 17 of 19 curated context–pathway pairs, compared with one for control mean and eight for control variance. Ten were unique to the learned stages (Table A6). Our conclusion is that the learned ranks recover known annotations missed by the expression baselines tested here. A comparison with expression deltas or the pretrained backbone alone remains open.
Thank you very much for your exceptionally detailed and thoughtful review! Two of your suggestions were especially helpful. We were able to run LPM on all four contexts, and using the measured panel as the GO background changed our interpretation of the RNA/protein result.
Q1. Baselines and metric calibration
Following your suggestion, we ran a separate pseudobulk comparison with LPM and report the results here. For each perturbation, we averaged Looped X-Cell’s predictions across cells so that both models could be evaluated at the same resolution. Across all four Replogle–Nadig contexts, LPM achieved lower mean MAE, while Looped X-Cell achieved lower mean weighted MSE (wMSE) and higher mean Pearson Δ and centroid accuracy.
Values are mean ± sample s.d. across five seeds (n=5). The LPM learning rate was 0.005. Bold indicates the better mean for each context and metric. Following Miller, Mejia et al. [5], we included wMSE because it gives greater weight to genes that distinguish each perturbation from the other perturbations in the same context. Across all four contexts, LPM achieved lower MAE, while Looped X-Cell achieved 21–36% lower wMSE, indicating smaller weighted prediction errors on these perturbation-specific expression profiles.
In the manuscript, we added Control and scDFM to Table 1. Control returns the same reference-control cells for every perturbation. Its centroid accuracy was close to chance in all four contexts (0.5013–0.5018), even though its MAE was lower than Linear’s in three. Looped X-Cell achieved lower mean MAE and higher mean Pearson Δ than scDFM in all four contexts. For Pearson Δ, we subtract the same target-context control mean from the predicted and observed pseudobulk profiles, then correlate the two changes across genes (Appendix B.4).
Appendix B.3 now describes Linear, a mean-based baseline that transfers each perturbation’s average effect from the source contexts to the target context. Table A2 summarizes the training and inference settings for the models in Table 1 and the public release terms for STATE, STACK, and TxPert.
[5] Miller, Mejia et al. Deep Learning-Based Genetic Perturbation Models Do Outperform Uninformative Baselines on Well-Calibrated Metrics. https://doi.org/10.1101/2025.10.20.683304