- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a method based on norm-preserving rotation to learn cell representations and predict perturbation effects. The results look promising, and the analysis is very interesting. The authors also demonstrate their prediction effect based on wet-lab validation with a control group.
Critical strengths
This paper proposes a method based on norm-preserving rotation to learn cell representations and predict perturbation effects. The results look promising, and the analysis is very interesting. The authors also demonstrate their prediction effect based on wet-lab validation with a control group.
Critical weaknesses
I have some questions to check:
- The motivation is not clear to me. Choosing from matching does not seem like a very strong method novelty in this area, and I also cannot see the strength directly. I am also not quite confident about the rotational operator, and it seems like cherry-picked geometric modelling, lacking solid theoretical support.
- How does preserving latent norm ensure that rotated cells remain within the decoder’s training distribution, especially if that distribution is anisotropic or contains distinct cell-state clusters?
- Because rotations generally satisfy , how does OpPert choose an ordering for simultaneous perturbations, and how sensitive are predictions to reversing that order?
- Since training includes measured combinations through the composition loss, how well does the Lie-bracket interaction score perform when the entire model is trained exclusively on single-perturbation data?
- How are the top-20 differentially expressed genes selected for uncertainty estimation on unseen perturbations? Does the confidence score remain useful when those genes must be selected without access to measured test responses?
- What additional experiments would establish whether the Belinostat-associated expression changes in PDE4D, PRKCA, and PRKCB contribute causally to apoptosis, rather than reflect downstream stress responses?
- More ablation studies and baselines should be included to defend the strength of this method; e.g., other methods might also produce wet-lab-strong results.
Questions for authors
See the weakness above
4 comments
Comment (2/4): Alternatives Tested and the Norm Constraint
Continuing from our first comment, we address the alternatives we tested and the norm constraint.
Alternatives we tested
…More ablation studies and baselines should be included to defend the strength of this method; e.g., other methods might also produce wet-lab-strong results.
-
To test the rotation against the operators one could choose instead, we trained a general linear control, , which has 512 parameters per perturbation against 192 for the rotation. Each block uses a full matrix generator and its matrix exponential, and to hold everything else fixed, we kept the encoder, decoder, descriptor encoder, losses, optimiser, schedule and split, and retrained the rotation backbone with the same code. For each operator, we trained three backbones differing in data order and dropout and selected each on the 13 validation perturbations. We report the mean ± population SD over these runs, each averaged over three flow seeds.
Norman top-20 DA in percent under the same training procedure
Operator Parameters 26 singles 79 pairs General linear 512 76.6 ± 4.5 91.3 ± 1.0 OpPert rotation 192 82.4 ± 2.0 93.0 ± 0.3 -
As the table above shows, the general linear control scores below the rotation in each of the three training runs despite 2.7 times the parameters, and a two-sided signed-rank test over the 105 perturbations gives . Its accuracy also varies more between runs, whereas the rotation reaches nearly the same accuracy in every run. The gap is largest on the held-out singles, where the flow predicts every generator for an unseen gene, which agrees with the error bound of our first comment.
-
Moreover, Section 4.6 replaces the rotation with the vector-additive latent shift of scGen, CPA and GEARS under the same encoder, decoder, flow and descriptor encoder, and Table 1 reports a rotation gain of 13 to 16 points in directional accuracy [L338-339, L450-457]. Within the rotation family, Table 11 finds the highest pair accuracy at , since makes every bracket vanish and larger blocks add parameters that the 70 training singles cannot supervise [L1594-1602].
-
In addition, we train and evaluate every learned method in Table 1 on the GEARS-standard splits under one pipeline, and Appendix F reports the sensitivity to nine further hyperparameters [R288-289, L462-466]. The RT-qPCR tests whether OpPert's predictions transfer to an independent experimental modality, rather than comparing methods, which Table 1 does on held-out data [L419-421].
Prompted by this comment, in the revised manuscript we have now added the general linear control to Appendix N.3 and Table 28 [R2299-2318]. The code is in the supplementary, where the rotation_general_linear setting in oppert/model.py switches every block to GL(4), and scripts/score_gl4_control.py reproduces the table from results/norman_gl4_control_per_perturbation.csv.
Norm and the decoder's training distribution
…How does preserving latent norm ensure that rotated cells remain within the decoder's training distribution, especially if that distribution is anisotropic or contains distinct cell-state clusters?
- We also considered this case when we chose the operator, and in our setting the learned rotations are small, with nearly all eigenangles below , which limits how far a rotation moves a latent state from its starting point [L319-321]. In addition, a rotation keeps the norm of every latent state it acts on, as Lemma 3 shows [L966-968]. However, in general, norm preservation does not constrain the direction of a latent state, as the anisotropic and clustered cases in your question show.
In the revised manuscript, we have now stated the guarantee in terms of the norm of the decoder input, in the introduction, Section 3.2, the contribution statement, the Figure 2 caption and Appendix B.5, to be more precise and avoid confusion [R049, R079, R148-149, R178-179, R1026-1030].
Comment (3/4): Pair Order and Training on Single Perturbations
Continuing from our second comment, we address the order of the two genes and training on single perturbations.
Order of simultaneous perturbations
…Because rotations generally satisfy (R_pR_q \neq R_qR_p), how does OpPert choose an ordering for simultaneous perturbations, and how sensitive are predictions to reversing that order?
-
This is an interesting point for discussion, since simultaneous perturbations have no intrinsic order and exchanging the two genes reverses the sign of their Lie bracket , as . However, this cannot change the interaction score in our case, as Eq. 4 uses only the norm of the bracket, which is the same in either order [L213-220]. For the prediction itself, we use alphabetical gene order during training and evaluation, which defines one prediction per pair.
-
To measure the sensitivity you ask about, we exchanged the genes in each held-out Norman pair with the checkpoint and flow seeds unchanged, and to isolate the bracket contribution, we restored alphabetical order and set at inference [R1766-1770, R2283-2289]. The table reports directional accuracy, DA, on the top differentially expressed genes, together with the -value of a two-sided Wilcoxon signed-rank test over the 79 pairs, each averaged over the three flow seeds and excluding zero differences.
Norman pair DA in percent from one trained model
DE genes OpPert Exchanged order OpPert minus exchanged p OpPert minus p Top-20 92.62 92.45 92.81 +0.17 0.56 −0.19 0.48 Top-50 92.48 92.22 92.42 +0.26 0.50 +0.06 0.92 -
As the table above shows, neither change makes a significant difference to pair DA at either cutoff within this trained model, which suggests that the ordering convention and the bracket scale have no significant effect on the accuracy that OpPert reports, consistent with Section 3.3 [R213-214]. We expect this, since both changes modify only the second-order term of the generator, which is small for most learned blocks [L205-212, L319-321]. Moreover, setting leaves the interaction score of Eq. 4 unchanged, because the score contains no [L1722-1723].
Prompted by this comment, in the revised manuscript we have now i) stated the alphabetical order in Section 3.3, ii) reported the paired test of in the caption of Table 17, and iii) added both paired tests to Appendix N.2 and Table 27 [R214-216, R1766-1770, R2283-2298].
Training on singles only
…Since training includes measured combinations through the composition loss, how well does the Lie-bracket interaction score perform when the entire model is trained exclusively on single-perturbation data?
-
In OpPert, training uses the 49 measured combinations of the GEARS training split, which calibrate through the training pair generator of Appendix N.6, whereas we hold out the 79 evaluated pairs, which never contribute to this gradient [L1374-1375, R254-261, R2353-2363]. We therefore compute each score without supervision from the candidate pair, as the abstract states [L018-019].
-
As suggested, we also retrained the backbone with every training combination cell removed, which leaves the 70 training singles and the control cells, with the rest of the training procedure unchanged. The bracket scale then stays at its initial value of 1, since only combination cells train it.
Norman top-20 DA in percent with and without training pairs
Training data 26 singles 79 pairs Singles only 81.0 ± 0.5 92.1 ± 0.3 Singles and the 49 training pairs 82.4 ± 2.0 93.0 ± 0.3 -
As the table above shows, the model trained without any pair reaches nearly the same accuracy on the 79 held-out pairs, and a two-sided signed-rank test over pairs finds no significant difference (). Therefore, OpPert predicts held-out pairs without any pair in training. In this model, BCH composes every held-out pair from its two single rotations with the analytic bracket scale , which shows that the composition rule itself, rather than pair supervision, produces the pair predictions. The bracket score of Eq. 4 needs only the two single generators, which the model trained on singles also provides [L213-220].
In the revised manuscript, we have now added this training on single perturbations to Appendix N.4 and Table 29 [R2319-2332].
Comment (4/4): Confidence Genes, Causal Role and Ethics
Continuing from our third comment, we address the genes of the confidence score, the causal role of the genes and the wet-lab ethics.
Genes for the uncertainty score
…How are the top-20 differentially expressed genes selected for uncertainty estimation on unseen perturbations? Does the confidence score remain useful when those genes must be selected without access to measured test responses?
-
In OpPert, we obtain the confidence signal from the flow itself, since the spread of the decoded expression shifts across independent flow draws measures how uncertain the flow is about an unseen perturbation, and Eq. 7 takes their coefficient of variation [L239-242, L376-378, L402-405]. A perturbation with a larger coefficient of variation therefore receives a lower confidence.
-
In Figure 5, we evaluate this score on the top-50 DEGs of each perturbation, which come from the measured response and match the genes on which we compute the directional accuracy shown on its y-axis [R397-400, R414-415]. However, we do not use that response to predict any unseen perturbation, and only the choice of genes for scoring depends on it.
-
To compute the score without any measured held-out expression, as suggested, we recomputed Eq. 7 on all genes for K562 and Norman, and on the 50 genes with the largest predicted change for Norman, since neither set requires measured held-out expression [R2058-2062]. The table reports the Spearman correlation of each score with prediction error, SPCC, the DA of the most confident 10% and 50% of targets, and the rank agreement of each score with the Figure 5 score.
Confidence score in percent for 270 K562 and 105 Norman targets
Dataset Gene set SPCC DA at 10% coverage DA at 50% coverage Rank agreement K562 Figure 5 top-50 DEGs 47.5 85.3 83.2 100 K562 All genes 39.3 85.0 81.9 98.1 Norman Figure 5 top-50 DEGs 30.6 98.4 94.5 100 Norman All genes 25.9 98.2 93.7 97.1 Norman Top-50 predicted 25.2 98.0 93.5 97.5 -
As the table above shows, the score retains a positive association with prediction error on every gene set, and the most confident predictions reach the highest accuracy on both datasets, also when we compute the score on gene sets that use no measured held-out expression. In addition, these scores rank the targets almost exactly as the Figure 5 score does.
Prompted by this comment, in the revised manuscript we have now stated the gene set of the score in the Eq. 7 text and the Figure 5 caption, and added this analysis to Appendix N.5 and Table 30 [R397-400, R414-415, R2334-2350], with the per-target scores in the supplementary files uncertainty_per_pert.json and norman_uncertainty_top50_predicted_degs.csv.
Causal role of the expression changes
…What additional experiments would establish whether the Belinostat-associated expression changes in PDE4D, PRKCA, and PRKCB contribute causally to apoptosis, rather than reflect downstream stress responses?
- That is a good point for discussion, since PRKCA and PDE4D regulate apoptosis in MCF7 cells, where knocking down PKCα induces apoptosis [5] and inhibiting PDE4D triggers apoptosis through cAMP and ER-stress signalling [6]. Their significant change under Belinostat is consistent with an adaptive survival response that accompanies the apoptotic phenotype [L426-430]. Loss-of-function or overexpression of these genes under Belinostat, with an apoptosis readout, would test their contribution directly, whereas our RT-qPCR tests whether OpPert's predictions transfer to an independent experimental modality [L419-421].
Ethics of the wet-lab stage
…Since they have wet lab results, I think it will be helpful to check the wet lab stage and if it fits the ethics rules.
- The study used the commercial human cell line MCF7, ATCC-HTB-22, authenticated by STR analysis, and it involved no human participants, no patient material and no animals [L2157-2158].
In summary, your detailed comments led to additional analyses, which show that i) a general linear control with more parameters scores below the rotation in every training run, ii) the order of the two genes has no significant effect on pair accuracy, iii) OpPert predicts held-out pairs without any pair in training, and iv) the confidence score ranks predictions without measured held-out expression. We are grateful for these suggestions, which have led us to i) state the norm guarantee and the gene order precisely, ii) add Appendix N and iii) extend the supplementary.
We hope these clarifications and revisions address your concerns, and we would be glad to answer any further questions.
[5] Cui et al. Fundam. Clin. Pharmacol. (2019).
[6] Mishra et al. Clin. Cancer Res. (2018).
Comment (1/4): Motivation and Theoretical Support for the Rotation
Dear Reviewer DM9H,
Thank you for the positive and constructive review. We address each point below, beginning with the motivation and the theory behind the rotation. Line references [L] point to the submitted paper and [R] to the revised paper.
Motivation and novelty
Why the rotation, and its theoretical support
[1] Sun et al. ICLR (2019).
[2] Su et al. Neurocomputing (2024).
[3] Arjovsky et al. ICML (2016).
[4] Lezcano-Casado et al. ICML (2019).