- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
REGINA predicts perturbation responses from unpaired control and perturbed single-cell populations. It combines a Transformer autoencoder, regularization based on cell-state labels, and a latent CycleGAN conditioned on a prompt derived by replacing the target gene’s expression with −1. The paper evaluates local prediction and distributional metrics on the Adamson, Replogle, and Norman datasets, and reports competitive differential-expression accuracy and improved distribution matching on selected benchmarks.
Critical strengths
1 The paper tackles an important setting in which matched cell-level outcomes are unavailable, and evaluates distributional properties alongside differential-expression accuracy.
2 Evaluation spans datasets and perturbation types, with baseline comparisons, ablations, a split-stability analysis, and a reported code release.
Critical weaknesses
1 The paper motivates REGINA as counterfactual prediction for individual cells, but unpaired distribution matching and cycle consistency do not identify a unique cell-to-cell mapping. The reported metrics assess aggregate distributions and gene-level effects, not whether a particular control cell’s predicted response is correct. The claims should be framed more explicitly around population-level matching unless additional evidence supports cell-level correspondence.
2 The distributional results are mixed across datasets and metrics. On Adamson, REGINA is worse than PerturbNet on MMD (1.53 vs. 0.56), Wasserstein distance (0.255 vs. 0.210), and variance correlation (0.43 vs. 0.89; Table 2). On Norman, it has the best MMD, but CPA is better on Wasserstein distance, variance correlation, and energy loss (Table 4). The small MMD gap on Norman is also modest relative to the reported variation. These results do not support a general improvement in distributional fidelity.
Questions for authors
Is the intended target a population-level distribution, or a valid counterfactual for each individual cell? Please clarify which claims rely on cell-level correspondence.
For Table 6’s “0/2 Seen” cases, what exactly is held out, and how many combinations are evaluated? Are both constituent single-gene perturbations absent from training?
Are the comparisons in Tables 2 and 4 paired on the same held-out perturbations? Please clarify how the reported variability affects the claimed advantages.
2 comments
I thank the authors for their response. I will keep my score.
We thank the reviewer for the comment and agree that the original phrasing caused confusion. The metrics used for evaluation indeed calculate the population-level distribution similarity between the target and predicted cell populations and the individual effects were evaluated during the training phase. We have clarified this in the revised manuscript.
For Table 6, we evaluated our model's performance on perturbation combinations. The 0/2 seen means that neither perturbed gene was seen during the training (in neither phase). We used 139 conditions for training, where condition means any individual gene perturbation or combined perturbation. For validation we used 31 conditions and for testing we used 107 conditions.
Regarding Tables 2 and 4, the held-out perturbations were exactly the same across all models for a given dataset (Adamson, Replogle, and Norman). The table grouping is based on precision versus distributional metrics, and single-cell versus multi-gene perturbation datasets.
To address the variability in the distributional results: our core claim is not that our model is strictly superior across all distributional metrics. Rather, we emphasize that REGINA achieves high precision while maintaining competitive (and sometimes superior) distributional fidelity, whereas other models often excel in one aspect but fail significantly in the other.