- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Authors have proposed scTREAT which uses latent diffusion to predict pseudobulk experssion profiles after drug conditioning.
They use a frozen scimilarity AE to define the latent space and the conditions are fused into 1 token before being injected into the denoiser.
Training uses an infoNCE regularizer. Inference uses CFG with separate state and drug representation coefficient.
Eval is done on unseen pairs and seen drugs benchmark. The authors report best values on 10-11 metrics against 8 baselines.
Ablations are also reported with qualitative analysis using UMAP representation and gene set enrichment.
Critical strengths
the datasets are clean and leakage is managed properly. DMSO controls are plate matched and unseen drugs held out set is across all doses and plates.
pathway analysis is correct in showing partial agreement.
Critical weaknesses
the metrics benchmarks are old. cell-eval 2 package has been released by ARC which calculate correctly calibrated metric values.
methods like CellFlow and scDFM are not reported which are current state of the art.
simple baselines like control + mean training drug difference within cell lines is not reported.
all results seem to be single runs. i did not see the word random seed or multiple seeds anywhere. always report mean + standard deviation on multiple seeds. confidence intervals are also important for baseline comparison and ablations since some wins are 0.01 to 0.02 margins.
cross attn to single condition token reduces to an input independent learned bias since softmax over one key equals 1. please explain the "non concatenative attention injection" claim in more detail for the reader.
I believe the FiLM layer is doing the same job as cross attention in here since output is condition dependent vector added to every token. please do the ablation and show results.
The biggest issue is that the contribution and novelty is extremely narrow. Latent diffusion and multi condition compositional guidance is not new, and just applying it to perturbation modelling with fixed dose to strength map is not the level of novelty for an ICLR paper.
Questions for authors
-
please reported calibrated metrics (maybe from cell-eval2) like perturbation discrimination, DE overlap/score, MAE relative to a mean-response baseline with multiple seeds and confidence interval. a control + mean delta baseline is also important. this will invoke more confidence for the reader.
-
what do ablations of L align or L info and of FiLM only conditioning (no cross-attention) show, given that cross attention to one token cannot depend on the latent?
-
How were sp, s0, alpha, gamma chosen, on which split, and how many zT draws per condition enter the metrics?
i believe this paper would be good for a computational biology venue rather than ICLR. the field of perturbation modelling has moved really quickly in the past year and incremental methods may show better numbers without any true ML novelty.