- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
this paper proposed genescope with adaptive resolution transformer for single-cell chemical perturbation prediction. it dynatically determines and select genes that should be kept under higher resolution representation without relying on unknown treatment response, based on features of control cell population. genescope contains following major modules for representation learning, 1) a macro encoder to compress gene represetntation for 2k input genes via three gene partition views, 2) a micro encoder to select 200 genes via control gene variance and reconstruction error and build fine res representation via attention mechanism, and 3) a population encoder to learn context across control cells via transformer. then the response decoder generates post-treatment expression profile using shared affine response, nonlinear control-mean correction and gene-specific residual prediction. main results show genescope get better improment on response Pearson on Tahoe dataset, and improved high-response gene retrieval on cellxgene-sciplex3 dataset.
Critical strengths
- clean paper writing explains the method and model clearly, with explaination on choice of architecture, objective and experiment settting, as well as revealing the limitations brought by baseline protocol difference, dataset dependence, etc.
- separately handle res allocation and treatment response prediction, to avoid using treated outcome for inference time selection, with dense macro mechanism gauranteeing response can come from any gene.
Critical weaknesses
- lack comparison for control-guided selection method to other task-specific/statistic-driven/general feature selection method.
- micro-only interventation removes macro attention and gene features, as well changed micro cross-attention memory, population pooling and gene selection score, thus cannot be used to differentiate , thus cannnot prove the lift comes from proposed adapative selection strategy.
- too many discrepancies exist between baseline setting and genescope prediction setting, e.g. in normalization, control sampling, profile used, etc., which greatly prevent me to trust the good looking numbers.
- high-response gene retrieval targets were calculated from observed response magnitude rather than FDR-controled DEGs which cannot prove compound-specific effects are truely identified from post-treatment population distribution.
Questions for authors
- can u provide direct evidence to prove control-guided selection criterion is core to corresponding performance improvement? e.g. under same genescope structure, k (=200), objective and compute resource, directly compare to random/variance-only/reconstruction-error-only/fixed seletion?
- can u rerun benchmark for ridge, chemCPA and STATE-ST under unified evaluation criterion?
- how did u handle the coverage limitation of adapting chemCPA on unseen treatment prediction?
- can you try to explain the better performance over ridge regression on tahoe-100m but degenerating on the tow sciplex data on unseen compound prediction?
We appreciate your suggestions for stronger controls and more consistent baseline comparisons. Our responses below present the new experiments and explain how they will inform the revision.
Response to Q1:
Thank you for your constructive suggestion. Control-guided selection improves mean Pearson and AUPRC over random and fixed panels; single-criterion rules remain competitive (Table 1). Following your suggestion, 36 runs compare four alternatives across three datasets and three matched seeds, fixing architecture, K=200, losses, initialization, split, 5,000 updates and four GPUs. Random samples one panel per seed; Fixed uses one training-control variance-ranked panel across conditions and seeds.
For W1-2, selection alone changes; Macro features, cross-attention and pooling remain fixed, unlike Micro-only.
Table 1. Response Pearson correlation on all test conditions, mean (own sample SD).
Eighteen further runs retain only energy distance, response MSE, reconstruction and selection stability. GeneScope selection still exceeds random in Pearson and AUPRC across datasets (e.g., SciPlex-middle Pearson 0.0804 vs 0.0700). Auditing 54 records showed that control-guided selection favored frequently detected genes.
Response to Q2:
We reran evaluation with common expression units and reference populations. GeneScope leads on Tahoe Pearson and SciPlex RMSE; ridge and STATE-ST have higher SciPlex Pearson (Table 2).
Addressing "normalization, control sampling", models share 2,000 output genes and reference means from all control/treated cells. Population models receive the same 256 controls, processed in two 128-cell groups by STATE-ST. Scores from raw predictions are averaged equally across conditions. We tuned ridge over seven regularization values on validation data, regenerated predictions from six saved STATE-ST models and retrained chemCPA (Q3). Training budgets and molecular/dose inputs remain model-specific; ridge adds training-control means.
Table 2. Means on test conditions with predictions from all models; STATE-ST limits coverage (counts). Space permits summary scores.
Your distinction in W4 is helpful: AUPRC evaluates recovery of genes with large observed changes, not FDR-controlled DEGs or causal specificity. A large, noisy change can rank highly without statistical significance. We have not run replicate-aware DEG tests, and the revision will clarify this and include the protocol and full results.
Response to Q3:
The original chemCPA adapter used compound-specific dose parameters and excluded compounds outside its training vocabulary. We addressed this restriction by replacing those parameters with a dose network shared across compounds and representing drugs using frozen Morgan256 fingerprints. An unseen compound can therefore enter through its molecular structure and dose, without fitting new drug-specific parameters. We also recovered 97 missing structures from chemical identities, without using test expression.
We then retrained chemCPA with three seeds per dataset for 100 epochs, selecting checkpoints on validation data. These nine models cover all test conditions, including unseen compounds. On unseen compounds in familiar cell lines, GeneScope outperforms chemCPA on response Pearson, AUPRC and RMSE on Tahoe, while chemCPA performs better on cellxgene-SciPlex3. We will document the adapter changes and include both coverage and transfer results in the revision.
Response to Q4:
Our interpretation is that limited cellular diversity in SciPlex makes state-dependent corrections harder to learn, favoring ridge's regularized chemical-to-response mapping for unseen drugs. Tahoe spans 50 cell lines; both SciPlex versions represent the same three-cell-line screen. For example, untreated A549 cells reveal the starting state but not an unseen compound's mechanism.
With unified scoring, Tahoe's unseen-compound advantage over ridge is in AUPRC (0.401 vs 0.365), while Pearson is slightly lower (0.441 vs 0.460). We have not isolated cell-line diversity from differences in compounds and preprocessing.