- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PerturbRx, a two-stage framework that transfers knowledge from single-cell perturbation data to patient drug-response prediction. Using frozen scFoundation and ChemBERTa representations, it learns drug- and dose-conditioned latent transitions from context-matched but unpaired control and treated cell populations in Tahoe-100M. Training combines distributional alignment with constraints on transition direction and magnitude. The learned transition model is then frozen and applied to tumor profiles, with the resulting treatment-conditioned features combined with baseline tumor and drug representations to train a response classifier without requiring matched post-treatment measurements. Experiments assess generalization to drugs excluded from source-stage training and response prediction on TCGA-485 and an independent PDX cohort. PerturbRx achieves the highest pooled out-of-fold performance among the evaluated methods, although its incremental gains over strong patient-state-plus-drug controls are modest and not statistically significant.
Critical strengths
The paper provides a useful connection between single-cell perturbation modeling and patient drug-response prediction. The evaluation combines drug-held-out tests, patient- and model-disjoint splits on TCGA and PDX, and representation and architecture controls, strengthening the empirical assessment of transferability and the added value of the learned transition features.
Critical weaknesses
The gains over controls using only patient and drug representations are small and not statistically significant, leaving the incremental predictive value of the learned transitions uncertain. Without matched post-treatment patient measurements, it also remains unclear whether the transferred features capture patient-specific biological responses or primarily provide a useful nonlinear transformation of existing representations.
Questions for authors
Are the gains over the Patient+Drug baseline consistent across training runs with different random seeds? Could you also compare with a frozen, randomly initialized transition network of the same architecture to clarify whether perturbation pretraining provides additional value beyond a nonlinear transformation of the existing representations?
4 comments
Response
Thank you for the additional experiments and clarifications.
The random-seed analysis changes my assessment of the original results. PerturbRx outperformed Patient+Drug in only 35/100 TCGA runs and 38/100 PDX runs, with slightly lower average AUROC on both datasets. The original gains therefore do not appear robust.
The revised ensemble improves some comparisons, but its advantage over the matched frozen-random transition network remains nonsignificant after multiple-testing correction. It is still unclear how much the perturbation pretraining contributes beyond adding a nonlinear feature transformation. The unseen-drug results also show mixed gains over simpler models.
I appreciate the new patient-level validation and the authors’ clear discussion of these limitations. However, the additional results have not convinced me that the main contribution provides a sufficiently reliable improvement in patient drug-response prediction.
Follow-up Response to Reviewer jyzj
We thank the reviewer for the follow-up. We acknowledge that the original single-head results were sensitive to initialization and that the specific predictive contribution of perturbation pretraining remains uncertain across benchmarks and metrics.
However, PerturbRx's broader contribution is an inductive framework that transfers treatment-conditioned representations from single-cell perturbations to unseen patient–drug pairs. Under our revised ensemble evaluation, PerturbRx significantly outperformed all four evaluated external methods (CODE-AE, DeepSADR, TransDRP, and WISER) in AUROC on both TCGA and PDX, and in AUPRC on PDX.
The revised ensemble also showed significant improvements over Patient+Drug on TCGA AUROC (Δ = +0.0215, p = 0.0031) and PDX+Drug on PDX AUPRC (Δ = +0.0303, p = 0.0027).
Beyond downstream prediction, we independently evaluated the frozen transition operator against observed treatment-associated molecular changes. We demonstrated directional transfer on sci-Plex 3 and improved agreement with observed changes in 25 of 27 independent patient cohorts.
We therefore emphasize the transferability of treatment-conditioned representations and their independent empirical validation, while acknowledging that the incremental predictive contribution of perturbation pretraining is not uniformly established.
Additional Analysis: Robustness to Frozen-Random Operator Initialization
We thank the reviewer for the follow-up. We conducted an additional analysis to assess whether the frozen-random comparison depended on a particular random initialization.
Additional analysis: 20 frozen-random operator initializations
Our original frozen-random control used a single operator initialization per benchmark. We therefore evaluated 20 independently initialized frozen-random transition operators on each benchmark under the same downstream ensemble protocol.
Performance across operator initializations
| Benchmark | Metric | PerturbRx | Frozen Random (20 runs, mean ± SD) | PerturbRx wins |
|---|---|---|---|---|
| TCGA-485 | AUROC | 0.7256 | 0.7121 ± 0.0021 | 20/20 |
| TCGA-485 | AUPRC | 0.8040 | 0.7835 ± 0.0024 | 20/20 |
| PDX-791 | AUROC | 0.696 | 0.6899 ± 0.0018 | 20/20 |
| PDX-791 | AUPRC | 0.620 | 0.5992 ± 0.0031 | 20/20 |
Paired cluster-bootstrap comparisons (PerturbRx minus mean Frozen Random)
| Benchmark | Metric | Δ | 95% CI | p-value |
|---|---|---|---|---|
| TCGA-485 | AUROC | +0.0135 | [-0.0069, 0.0339] | 0.1933 |
| TCGA-485 | AUPRC | +0.0206 | [-0.0059, 0.0469] | 0.1294 |
| PDX-791 | AUROC | +0.0059 | [-0.0059, 0.0178] | 0.3318 |
| PDX-791 | AUPRC | +0.0204 | [0.0020, 0.0387] | 0.0302 |
Paired cluster bootstrapping was performed at the patient level for TCGA-485 and at the PDX-model level for PDX-791, accounting for repeated drug-response episodes within clusters.
PerturbRx outperformed all 20 frozen-random initializations on both metrics and benchmarks, indicating that its numerical advantage was not driven by an unusually weak random initialization. The paired bootstrap comparisons showed a statistically significant improvement for PDX AUPRC (p = 0.0302), but not for the other three benchmark-metric combinations.
These findings strengthen the evidence for initialization-robust numerical improvements and suggest an incremental contribution from perturbation pretraining on PDX AUPRC, while leaving its consistent statistical advantage across benchmarks and metrics unresolved. We have updated the manuscript accordingly.
Response to Reviewer jyzj
We thank the reviewer for the constructive suggestions. The concern about random-seed variability motivated an additional robustness analysis, which revealed substantial initialization sensitivity and informed our revised downstream ensemble design. We also added the requested frozen-random transition control and validation against observed patient treatment-associated changes.
1. Incremental predictive value and random-seed robustness
Following the reviewer's suggestion, we evaluated 100 matched random initializations under the original single-head configuration using identical outer folds (Appendix C.4, Tables 22–23).
The original gains over Patient+Drug were not consistently reproduced: PerturbRx achieved higher AUROC in only 35/100 TCGA and 38/100 PDX runs. Across these runs, mean AUROC was 0.691 versus 0.696 on TCGA and 0.674 versus 0.679 on PDX (PerturbRx versus Patient+Drug, respectively).
This initialization sensitivity motivated a multi-architecture, multi-seed response-head ensemble. The revised predictor averages predictions from six response-head configurations across 20 random seeds, reducing reliance on any single initialization. We evaluated the revised protocol using the same grouped outer folds and corresponding representation controls.
Under this protocol, PerturbRx achieved pooled OOF AUROC/AUPRC of 0.726/0.804 on TCGA and 0.696/0.620 on PDX, compared with 0.704/0.786 and 0.692/0.589 for Patient+Drug and PDX+Drug, respectively.
The TCGA AUROC improvement (Δ = +0.0215, p = 0.0031) and PDX AUPRC improvement (Δ = +0.0303, p = 0.0027) were statistically significant. The PDX AUROC difference was not significant.
2. Frozen random transition control
Following the reviewer's recommendation, we evaluated an architecture-matched, randomly initialized transition network with frozen weights under the revised ensemble protocol.
PerturbRx achieved numerically higher AUROC/AUPRC than this control on TCGA (0.726/0.804 versus 0.715/0.785) and PDX (0.696/0.620 versus 0.688/0.597).
On PDX, the AUPRC difference was +0.0229 (95% CI: [0.0048, 0.0409], p = 0.0142), while the AUROC difference was not statistically significant.
These results suggest that perturbation pretraining may provide additional predictive value in specific settings, but they do not establish a consistent advantage attributable specifically to pretraining rather than an additional nonlinear transformation.
3. Biological validation
We additionally evaluated paired pre- and post-treatment transcriptomes from 278 patients across 27 independent cohorts.
Compared with a leave-one-patient-out Drug Mean baseline, PerturbRx improved directional agreement in 25/27 cohorts (92.6%) and 207/278 patients (74.5%), with a mean paired cosine gain of +0.0222 (95% CI: [0.0173, 0.0272]).
Improvements were observed in both bulk RNA-seq (19/20 cohorts) and scRNA-seq (6/7 cohorts). These findings provide empirical evidence of modest but broadly positive treatment-associated directional transfer, although absolute agreement remained limited and heterogeneous. Accurate patient-specific post-treatment reconstruction was not established.
We appreciate the reviewer's suggestions, which helped us identify downstream initialization sensitivity, strengthen the evaluation of learned versus untrained transitions, and clarify the biological interpretation of transferred representations (Sections 3.4, 4.3–4.5, and Appendix C.4).