- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the prediction of single-cell perturbation responses, aiming to forecast the expression distribution of control cells under previously unseen perturbations or combinations thereof. The authors propose a two-stage framework called DRIFT. In the first stage, a Variational Autoencoder (VAE)—incorporating a conditional prior and a mutual information penalty—decomposes each cell into an invariant component () and a response component (). In the second stage, a flow-matching model, conditioned on and the perturbation encoding, is employed to transform only the component. During inference, the components of the control cells are kept constant, while the components are generated.
Critical strengths
- The approach effectively combines the disentanglement of a "base state" (akin to the CPA model) with the flexible transformation capabilities of flow-based models; this methodology is readily transferable to other architectures.
- The motivation is intuitive. By separating perturbation-related information from background (contextual) information, the model can specifically focus on the perturbation-driven dynamics via flow matching.
Critical weaknesses
- The strategy of directly copying during inference implicitly assumes invariance at the single-cell level. However, the CLUB objective used only enforces independence at the population level, and the paper provides no empirical evidence to support invariance at the level of individual-cell counterfactuals.
- The paper lacks a sensitivity analysis for hyperparameters. The actual impact of , , and depends on the scale of the latent space—a detail not reported in the paper. Furthermore, the weighting for the perturbation encoding regularization is omitted, and the protocol for hyperparameter selection is not specified.
- The baselines are drawn from existing literature, yet the model is not compared against scDFM on the Norman dataset.
- The paper fails to cite scDRP, a framework that is highly similar in design.
Questions for authors
See my Weaknesses.
4 comments
The baselines are drawn from existing literature, yet the model is not compared against scDFM on the Norman dataset.
We follow scDFM's protocol, including the dataset splits, the number of generated cells per condition, and the evaluation metrics, and we reproduce the scDFM numbers from its official released predictions. scDFM's preprocessing keeps 5,029 genes for training, against 2,051 in the scBIG benchmarks. DRIFT keeps the architecture and objective used on the scBIG benchmarks, with hyperparameters selected on a validation split held out from scDFM's training data. Results for these new experiments are in the tables below:
| Additive (4 splits) | Pearson Δ ↑ | DE-Sp. ↑ | DS ↑ | L2 ↓ | MSE ↓ | MAE ↓ | Δ̂ ↑ | Δ̂₂₀ ↑ |
|---|---|---|---|---|---|---|---|---|
| Control | N.A. | N.A. | 0.514 | 3.971 | 0.01823 | 0.0394 | -0.191 | -0.152 |
| Additive | 0.905 | 0.604 | 0.966 | 1.913 | 0.00438 | 0.0224 | 0.859 | 0.925 |
| scDFM | 0.885 | 0.774 | 0.974 | 1.704 | 0.00315 | 0.0215 | 0.843 | 0.924 |
| DRIFT | 0.929 | 0.914 | 0.983 | 1.584 | 0.00295 | 0.0194 | 0.903 | 0.941 |
| Holdout, single (4 splits) | Pearson Δ ↑ | DE-Sp. ↑ | DS ↑ | L2 ↓ | MSE ↓ | MAE ↓ | Δ̂ ↑ | Δ̂₂₀ ↑ |
|---|---|---|---|---|---|---|---|---|
| Control | N.A. | N.A. | 0.523 | 2.688 | 0.00953 | 0.0263 | 0.155 | 0.188 |
| scDFM | 0.713 | 0.735 | 0.891 | 1.619 | 0.00301 | 0.0190 | 0.666 | 0.812 |
| DRIFT | 0.801 | 0.886 | 0.891 | 1.381 | 0.00233 | 0.0145 | 0.722 | 0.775 |
| Holdout, double (4 splits) | Pearson Δ ↑ | DE-Sp. ↑ | DS ↑ | L2 ↓ | MSE ↓ | MAE ↓ | Δ̂ ↑ | Δ̂₂₀ ↑ |
|---|---|---|---|---|---|---|---|---|
| Control | N.A. | N.A. | 0.532 | 4.193 | 0.02073 | 0.0424 | -0.206 | -0.100 |
| scDFM | 0.836 | 0.754 | 0.919 | 2.031 | 0.00471 | 0.0235 | 0.762 | 0.860 |
| DRIFT | 0.881 | 0.905 | 0.914 | 1.758 | 0.00383 | 0.0197 | 0.816 | 0.859 |
DRIFT obtains the best performance. In the additive setting it is first on all eight metrics, ahead of scDFM and the additive baseline. On held-out genes it is first on 12 of the 16 metrics, with a large margin on Pearson Δ (0.801 against 0.713 for held-out single genes). The performance of DRIFT therefore also holds under scDFM's protocol.
The paper fails to cite scDRP, a framework that is highly similar in design.
We thank the reviewer for the pointer. The revision will add it to the related work. scDRP is a disentanglement-based method that separates perturbation-responsive from invariant latent variables. It represents each perturbation as a one-hot index, transports cells with an optimal transport plan computed for each measured perturbation, and estimates the effect of a combination by vector addition of the individual treatment effects of each single perturbation. A one-hot encoding does not generalize to unseen genes, and an OT plan computed for each perturbation is less flexible than DRIFT's single flow, conditioned on gene and drug features and shared across perturbations. This allows DRIFT to predict unseen genes and non-additive combinations, which scDRP's one-hot and additive design does not directly support.
Bunne, C., et al. (2023). Learning single-cell perturbation responses using neural optimal transport. Nature Methods, 20(11), 1759-1768.
Tong, A., Fatras, K., Malkin, N., Huguet, G., Zhang, Y., Rector-Brooks, J., ... & Bengio, Y. (2023). Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint arXiv:2302.00482.
Thank you for the additional experiments. I remain unsure which biological features preserves across perturbations, so I lean toward keeping my original rating.
We thank the reviewer for the reply. In short, preserves the technical and compositional state of the cell (sequencing depth, ribosomal and mitochondrial content), while the biological programs are encoded mainly in and transported with the perturbation.
The initial concern was invariance at the level of individual-cell counterfactuals. As stated in our response, we do not claim per-cell counterfactuals, and they cannot be verified with unpaired data, since each cell is measured once. What can be analysed is which attributes of a cell encodes. We already reported the gem group in our response, and we provide further analyses below on the Norman additive dataset.
We use the attributes recorded for each cell in the dataset (total counts, detected genes, mitochondrial and ribosomal fractions, on which the prior of is conditioned), and scores computed from expression for the cell cycle and the 38 MSigDB hallmark programs. On held-out control cells, a regressor predicts each attribute from or from . Results of these new experiments are below:
- encodes the technical and compositional state of the cell, and a generated cell keeps that of its source control cell. The last two columns compare, within each perturbation, the attribute of a generated cell with that of its source control cell (5 seeds):
| Attribute | from | Generated vs source control cell | Generated vs random control cell |
|---|---|---|---|
| Total counts | 0.99 | 0.85 | 0.00 |
| Detected genes | 0.96 | 0.83 | 0.00 |
| Mitochondrial fraction | 0.98 | 0.79 | 0.00 |
| Ribosomal fraction | 0.99 | 0.91 | -0.03 |
- Biological programs are encoded in . Every program is predicted better from than from (5 seeds):
| Program | from | from |
|---|---|---|
| Cell cycle (G2M score) | 0.25 | 0.84 |
| E2F targets | 0.25 | 0.75 |
| MYC targets | 0.49 | 0.82 |
| Oxidative phosphorylation | 0.49 | 0.77 |
| Heme metabolism (erythroid) | 0.16 | 0.69 |
| p53 pathway | 0.11 | 0.29 |
| All 38 hallmark programs (mean) | 0.13 | 0.37 |
DRIFT transports these programs with the perturbation. For example, CDKN1C or TP73 increase the fraction of cells in G1 by 28 to 40 points, and on the 32 test combinations, the change in the G1 fraction predicted by DRIFT correlates with the observed change at 0.895 (5 seeds).
These analyses identify the features that preserves, and we will add them to the revised paper.
We thank the reviewer for recognizing the method's intuitive design and transferability. We address the concerns raised below.
We do not claim per-cell counterfactuals, which are not identifiable from unpaired snapshots (Bunne et al., 2023). DRIFT predicts the distribution of perturbed cells, which is what is measured by every metric we report. For this target, copying the invariant block of a control cell requires znr to have the same distribution in control and perturbed cells. This is the population-level independence that CLUB enforces. We will revise the wording in the paper to clarify this aspect.
Empirically, three results support the copy strategy on the Norman additive dataset:
Generated cells carry the gem group of the control cell they were transported from (37.9%, against 12.8% for a random control cell), close to the 41.4% the autoencoder keeps with no transport (5 seeds). This is consistent with the copied znr keeping information on the source cell through the transport.
Scale. The OT cost is normalized per block, with each term divided by its minibatch mean, so αnr, αr, and ε do not depend on the latent scale. We will add this to the explanation of Eq. (9).
Sensitivity. Norman additive, one value changed at a time over a 16-fold range (defaults αnr=αr=1, ε=0.5). The flow is retrained on the submitted stage-1 model. Results of these new experiments are below:
Every metric stays within about 0.02 of the default, and DRIFT stays above scBIG on all five metrics for every setting we explore. This agrees with OT-CFM (Tong et al., 2023), on which the pairing we propose builds, where any coupling yields a flow onto the same target distribution.
All settings were selected on the validation set, using several evaluation metrics jointly, such as ρΔ, PDS, and MSE.
For the perturbation-encoder regularization, we use λaux=λiso=10. These weights are already set in the code attached to the submission and will be added to the appendix (Table 5) with all other settings.