- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MoRe, a framework for predicting transcriptional responses to unseen drugs. It combines three drug representations (chemical descriptors, target identities, and UniPert embeddings) with an auxiliary condition-mean response supervision loss. Experiments on ChemCPA and PerturbDiff across sci-Plex3 and Tahoe-100M demonstrate improvements in perturbation-response prediction. Additional ablations and latent-space analyses investigate the contributions of multimodal fusion and response supervision.
Critical strengths
The paper addresses an important challenge in perturbation modeling: generalization to unseen drugs. Integrating complementary chemical and biological representations is well motivated, and explicitly supervising perturbation-induced responses is a reasonable approach. Evaluation across two model architectures and datasets, together with extensive ablation studies, provides a useful empirical foundation.
Critical weaknesses
Weakness 1: The empirical evidence does not convincingly establish the necessity or effectiveness of the proposed framework.
a. Marginal improvements over stronger baselines. The headline 113% improvement is measured against a weak ChemCPA baseline. More meaningful comparisons reveal substantially smaller gains: on sci-Plex3, ChemCPA with UniPert and response supervision achieves 0.247 versus MoRe's 0.249, while PerturbDiff improves only from 0.280 with fusion alone to 0.286 with the complete framework. These differences are not convincingly established as statistically reliable.
b. Insufficient justification of the proposed mechanism. The theoretical analysis relies on a simplified squared-error objective that does not directly characterize the actual backbone losses. Moreover, the large observed within-condition variance does not necessarily imply gradient-level signal dilution.
The paper does not convincingly demonstrate that its two components jointly resolve the claimed fundamental limitations, substantially weakening its methodological contribution.
Weakness 2: The evaluation does not convincingly demonstrate genuine out-of-distribution generalization or preserved biological fidelity.
a. Questionable generalization setting. Held-out drugs are represented using external target annotations and pretrained embeddings, while evaluation relies on fixed random drug splits. Without scaffold-, target-, or mechanism-disjoint evaluations, improvements may primarily reflect interpolation through known biological relationships rather than genuine extrapolation.
b. Potential degradation of cellular distribution modeling. Despite improvements in DEG-based metrics, MoRe worsens PerturbDiff's all-gene energy distance from 0.537 to 0.566. This challenges the claim that response supervision preserves distributional fidelity.
These limitations directly undermine the paper's central claim of improving practically meaningful unseen-drug prediction.
Questions for authors
Q1. Given the marginal improvements over strong single-view and fusion-only baselines, can the authors demonstrate statistically significant and consistent benefits of the complete framework across multiple independent drug splits?
Q2. How much of MoRe's improvement depends on external target annotations or pretrained representations? Can the authors evaluate scaffold-disjoint and target-disjoint generalization and clarify potential overlap between representation pretraining and test compounds?
Q3. Can the authors provide direct evidence for the proposed gradient-level signal-dilution mechanism and explain why improved condition-mean prediction is accompanied by degraded all-gene distribution metrics?
6 comments
Response to Reviewer PVVF (2/3)
Continued: W1b / Q3 (first part) and W1c.
W1b / Q3 (first part). The scope of the decomposition, and direct evidence for gradient dilution
Thank you for both parts of this point.
1. Measured directly. Following your suggestion, we measured the gradient that the drug latent receives, on trained models and with each backbone's own loss, rather than inferring dilution from the within-condition variance. For a randomly sampled training condition, we drew 16 resamples of 64 cells and took the gradient of each loss with respect to the drug latent. The ratio ‖mean gradient‖ / mean ‖gradient‖ is 1 when all resamples agree in direction and falls as sampling noise dominates. Under each backbone's native loss a substantial part of that gradient is noise; under response supervision it is largely coherent (Table P3).
Table P3. Gradient signal-to-sampling-noise ratio with respect to the drug latent (sci-Plex3; three seeds × three checkpoints; mean ± sd over the nine runs). Both losses are evaluated on the same checkpoints.
| Model | Native loss | Response supervision (ours) |
|---|---|---|
| ChemCPA | 0.49 ± 0.18 | 0.91 ± 0.10 |
| PerturbDiff | 0.71 ± 0.13 | 0.87 ± 0.15 |
The native gradient is noisier for ChemCPA (0.49) than for PerturbDiff (0.71), so response supervision changes more for ChemCPA. This is why it carries most of the gain for ChemCPA, while fusion does for PerturbDiff (W1c).
2. The decomposition. It is stated for squared error, and its role is to motivate the form of the auxiliary term, for which it is exact (Eqs. 6 and 7). The evidence that the native objectives leave the drug latent with a noisy gradient is Table P3 (Sec. 3.3; App. E).
Full results. App. E: the decomposition with its assumptions, and the gradient ratio per checkpoint.
W1c. "Jointly resolve the claimed fundamental limitations"
Thank you for raising this point. The revision now shows directly that each component addresses its limitation, and that the two are needed together:
- The training signal. The native objective gives the drug latent a noisy gradient, and response supervision a coherent one (Table P3).
- The drug information. With a view noisy or missing, + MoRe stays above the model that relies on that view alone (Table K5 in our reply to Reviewer k4EA).
- The two together. In the ablation, each component alone improves the backbone, and + MoRe is significantly better than fusion alone (p < 0.001) and than response supervision alone (p = 0.024) (Table P2 in our first comment; Table D4 in our reply to Reviewer dRXi).
Which limitation matters more depends on the backbone. ChemCPA lacks the training signal: its reconstruction objective captures little of the differential response, so response supervision carries the gain. PerturbDiff on sci-Plex3 lacks the drug information: it is expressive enough to exploit richer drug views, so fusion carries the gain. Addressing both is what lets one plug-in work on both backbones. The Introduction now makes this dependence explicit.
Response to Reviewer PVVF (3/3)
Continued: W2a / Q2 and W2b / Q3 (second part).
W2a / Q2. Interpolation or extrapolation; disjoint splits; dependence on external annotations
Following your suggestion, we evaluated scaffold-disjoint, target-disjoint, and scaffold- and target-disjoint splits on both datasets, five folds each (Table R1 in the general response). + MoRe improves over the original backbone on every split of both datasets.
- The gain does not come from interpolation. When the held-out drugs share no scaffold or no direct target with the training drugs, + MoRe still improves PerturbDiff by 10–21% in DEG50 Pearson, as much as on random folds, with statistical significance in five of the six disjoint settings.
- The gain does not depend on any single external source. With the target view or the UniPert view replaced by noise for all held-out drugs, + MoRe stays above the original backbone (Table K5 in our reply to Reviewer k4EA). With chemistry alone and response supervision, without any annotation or pretrained embedding, it still improves both backbones on both datasets (Table D4 in our reply to Reviewer dRXi; Tables 11–14).
- Overlap with UniPert's pretraining set does not explain the gain. UniPert is trained on compound–target pairs without expression data. The gain of + MoRe is at least as large for held-out drugs outside its pretraining set as for those inside it (Table D5 in our reply to Reviewer dRXi).
Full results. App. M: every split, all metrics. App. N: the overlap with the training drugs, by split.
W2b / Q3 (second part). All-gene energy distance rises from 0.537 to 0.566
Thank you for pointing this out. We repeated the single-cell evaluation on sci-Plex3 over five random-split folds, for both backbones and by dose (Table P4).
- + MoRe matches or improves the distribution fit in 19 of the 20 comparisons: significantly better in 7, not significantly different in 12.
- The one exception arises at low doses. The all-gene energy distance of PerturbDiff rises over all doses (+7%, p = 0.03), driven by the lower doses, where drugs have almost no effect (Hetzel et al., 2022). At the highest dose it does not rise, and four of the five metrics improve significantly.
- Response supervision constrains the condition mean, not the cell-to-cell spread. A strong effect shifts the mean, which + MoRe predicts better; under a weak effect the distance reflects the spread, which remains with the backbone's native objective. Pseudobulk-level prediction is also often more important for clinical discovery, since no cell is observed both before and after treatment.
The revised Sec. 5.1 states how response supervision acts on the mean and on the spread.
Table P4. Single-cell distribution metrics on sci-Plex3 over all doses and at the highest dose (mean over five random-split folds; 1,000 cells per condition). E-distance: energy distance; W1: mean gene-wise Wasserstein-1 distance; DE recall: share of the observed differentially expressed genes of a condition that are recovered from the predicted cells. Bold: better with MoRe. ∗: differs from the original backbone (paired t-test over folds, p < 0.05).
| Doses | Model | E-distance, all genes ↓ | W1, all genes ↓ | E-distance, DEG50 ↓ | W1, DEG50 ↓ | DE recall ↑ | |
|---|---|---|---|---|---|---|---|
| PerturbDiff | All | Original | 0.471 | 0.0266 | 0.371 | 0.0973 | 0.124 |
| + MoRe | 0.504∗ | 0.0267 | 0.368 | 0.0903∗ | 0.152∗ | ||
| Highest | Original | 0.628 | 0.0320 | 0.589 | 0.1557 | 0.192 | |
| + MoRe | 0.616 | 0.0307∗ | 0.530∗ | 0.1319∗ | 0.246∗ | ||
| ChemCPA | All | Original | 1.971 | 0.0820 | 1.180 | 0.2697 | 0.007 |
| + MoRe | 1.970 | 0.0829 | 1.177 | 0.2708 | 0.008 | ||
| Highest | Original | 2.039 | 0.0828 | 1.360 | 0.3220 | 0.012 | |
| + MoRe | 2.038 | 0.0836 | 1.345∗ | 0.3208 | 0.013 |
Full results. App. H: Tables 16 and 17.
References
- Hetzel et al. Predicting cellular responses to novel drug perturbations at a single-cell resolution. NeurIPS, 2022.
We greatly appreciate the time and effort you’ve put into reviewing our submission. We have carefully responded to all your concerns. We would be grateful if you could consider raising your score in light of the new experiments, explanations and revision. Thank you again for your time and thoughtful suggestions!
Best regards,
The Authors
Response to Reviewer PVVF (1/3)
We thank the reviewer for the precise and constructive critique, which led to a stronger test of each claim. We summarise the main changes first and respond to each question in detail below:
W1a / Q1. Margins over strong baselines; significance across splits
Thank you for asking this.
1. "Statistically significant and consistent benefits across multiple independent drug splits." (Q1) Yes. We repeated the experiments on five folds of four types of drug split: random, scaffold-disjoint, target-disjoint, and both. + MoRe improves the original backbone in every setting, with statistical significance in 8 of the 10 settings (Table R1 in the general response; Table P1).
Table P1. + MoRe minus the original backbone on five random-split folds: mean ± standard deviation of the paired difference over folds. ∗: paired t-test over folds, p < 0.05.
2. "Marginal improvements over stronger baselines." The two baselines you mentioned are each one component of MoRe. We retrained them on five random-split folds of sci-Plex3 (Table P2).
Table P2. + MoRe against each of its two components used alone, on sci-Plex3 (DEG50 Pearson; mean over random-split folds). + fusion: the three drug views, without response supervision. + supervision: response supervision with a single drug view, chemistry (C, the original view) or UniPert (U, the best single view). SNR: signal-to-noise ratio of the observed response on the DEG50 (median over held-out conditions). Bold: best of the row. ∗: significantly different from + MoRe (paired t-test over folds, p < 0.05).
Full results. Table 2 in the main text; App. M: all five metrics. App. H: every arm under both objectives.
References