- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents BASIL, a batch-aware framework for predicting transcriptional responses to unseen genetic perturbations using single-cell data. The authors identify two important challenges in perturbation-response modeling: experimental batch effects, which can confound biological responses, and the absence of paired control–perturbed cells due to destructive sequencing. To address these challenges, BASIL introduces a dual-resolution learning framework that combines batch-matched pseudo-bulk supervision with condition-specific distribution alignment. To support generalization to unseen genes, the framework further combines GenePT embeddings with a multi-relational biological graph encoder incorporating transcription-factor regulation, Gene Ontology similarity, and protein interactions. The evaluation covers three genome-scale CRISPRi datasets and the Norman combinatorial CRISPRa benchmark, including unseen-condition, one-unseen-gene, and two-unseen-gene settings. The paper reports strong performance relative to several statistical, graph-based, foundation-model, and distribution-aware baselines.
Critical strengths
S1. The paper correctly distinguishes matching experimental conditions from establishing one-to-one cellular correspondences. Its combination of batch-specific reference states and population-level regularization is well motivated by the structure of the available data.
S2. The framework provides a coherent connection between mean-response supervision and distributional modeling. This design avoids training completely separate mean-prediction and distribution-generation models. The pseudo-bulk objective provides a relatively stable target for perturbation-effect estimation, while conditional MMD offers an additional constraint on the predicted single-cell population.
S3. The evaluation includes meaningful out-of-distribution generalization settings. Unlike evaluations that merely hold out cells from previously observed perturbation conditions, BASIL evaluates unseen perturbation identities.
Critical weaknesses
W1. The methodological novelty is somewhat incremental relative to existing perturbation-prediction approaches. The proposed framework largely integrates several established ideas. E.g., 1) Batch-matched control referencing and biological knowledge graphs have been investigated in prior methods, including TxPert. 2) Pseudo-bulk supervision and direct prediction of population-level perturbation effects are established approaches. 3) MMD-based distribution alignment has previously been employed in single-cell perturbation modeling, including scDFM. 4) GenePT and relation-aware graph propagation provide an established mechanism for representing unseen genes.
The main contribution therefore appears to be the specific integration of these components through a shared response operator rather than a fundamentally new modeling principle. This integration is reasonable, but the paper should more clearly identify what cannot be achieved by simpler combinations of existing techniques. In particular, the relationship to TxPert and scDFM should be sharpened. An important question is whether the improvement primarily comes from stronger gene representations, improved batch referencing, or the proposed dual-resolution objective.
W2. The empirical contribution of conditional MMD is surprisingly small, and its intended distributional benefit is not directly evaluated. This is my primary technical concern. Conditional MMD is central to the paper's title and methodological motivation. However, Table 3 shows that removing conditional MMD decreases Pearson delta only from 0.688 to 0.684 on RPE1 and from 0.671 to 0.668 on HepG2. These gains are modest, particularly relative to the much larger contributions from GenePT features and pseudo-bulk supervision.
W3. The biological perturbation encoder is not sufficiently isolated experimentally. BASIL combines GenePT embeddings with three biological relation types: transcription-factor regulation, Gene Ontology similarity, and STRING protein interactions. The paper provides an ablation replacing GenePT with nonsemantic features, but this changes a major source of perturbation information rather than independently assessing the graph encoder. Important missing comparisons include GenePT without graph propagation, graph propagation with individual relation types removed, and simpler perturbation encoders with matched model capacity.
These experiments would clarify whether the biological graph contributes meaningful information beyond that already encoded by GenePT. This issue is particularly important because the strongest results occur under unseen-gene generalization, where pretrained biological representations are likely to play a central role.
W4. The proposed combinatorial representation is relatively simple and does not explicitly model genetic interactions. For multiple perturbations, BASIL averages their graph-encoded gene embeddings. Although this produces a permutation-invariant representation, it does not explicitly represent interactions between the constituent perturbations before fusion with the basal cellular state. This raises a question about the interpretation of the strong two-unseen-gene results.
W5. The experimental evaluation does not fully establish robustness to batch variation or generalization across biological contexts. Although batch awareness is a primary motivation, the ablation removing batch matching produces relatively modest performance changes: from 0.688 to 0.679 on RPE1 and from 0.671 to 0.654 on HepG2. These results indicate that matching can help, but they do not establish robustness under substantial batch shifts. A more direct experiment would evaluate performance across batches with different technical characteristics, including held-out batches or deliberately challenging batch-transfer settings.
Questions for authors
Q1. Since conditional MMD is a central component, can the authors report quantitative evaluation of held-out predicted cell distributions, rather than relying exclusively on mean-effect Pearson correlation?
Q2. Are the small improvements from conditional MMD statistically significant across the four random seeds? Please report standard deviations or confidence intervals, preferably for all principal comparisons.
Q3. How much improvement comes from multi-relational graph propagation beyond the pretrained GenePT representations? Please provide GenePT-only and relation-specific graph ablations.
Q4. How does BASIL perform on strongly non-additive gene combinations? Can the authors compare against a simple additive response predictor and report results stratified by genetic-interaction strength?
4 comments
W3-W5 Comment
W3: The graph encoder’s contribution beyond GenePT requires independent ablation. We evaluated graph variants on 274 held-out perturbations in the Jurkat unseen-gene setting, keeping GenePT embeddings, training samples, and evaluation targets identical. The no-graph control preserves encoder layers, node-wise transformations, and the nominal parameter count, while setting all relation messages to zero. The full three-relation model achieved higher clustering agreement than the no-graph and each single-relation control, improving by 2.40% over no-graph and 7.91%, 3.55%, and 2.53% over TF-only, GO-only, and STRING-only, respectively. This metric compares the clustering structures of predicted and observed perturbation-mean expression profiles, supporting an additional contribution from graph propagation beyond GenePT. Relation-removal ablations provide complementary evidence: removing TF or STRING increased MSE from 0.004337 to 0.004413 and 0.004384, respectively; retaining these relations reduced MSE by 1.73% and 1.09%. We will include the complete no-graph, single-relation, and relation-removal results and configurations in the revised manuscript and appendix.
| Perturbation encoder | Clustering agreement |
|---|---|
| GenePT + node-wise encoder, without graph messages | 0.2821 |
| GenePT + TF messages only | 0.2677 |
| GenePT + GO messages only | 0.2789 |
| GenePT + STRING messages only | 0.2817 |
| GenePT + TF, GO, and STRING messages | 0.2888 |
W4: Mean-pooled gene embeddings lack explicit pairwise interactions, requiring clarification of the strong two-unseen results. Averaging embeddings does not impose additive expression responses. Mean pooling constructs a permutation-invariant combination representation, which enters a state-dependent nonlinear response network:
z_{AB}&=\frac{h_A+h_B}{2},\qquad z_b=E_b(x),\\\\ \widehat{\delta}_{AB}(x)&=D_\Delta\\!\left(\Phi\\!\left([z_b;z_{AB};z_b\odot z_{AB}]\right)\right). \end{aligned}$$ Because $\Phi$ and $D_\Delta$ contain nonlinear transformations, the combination prediction is not constrained to equal the sum or average of single-gene predictions. The product $z_b\odot z_{AB}$ represents **state–perturbation interactions**, while the downstream nonlinear mapping can represent non-additive combination responses implicitly. In Norman’s **two-unseen setting**, neither target gene has perturbation-response examples in training, but its GenePT embedding and external biological relations remain available. These representations provide biological context for the shared response network to generalize to unseen combinations. BASIL achieves Pearson $\Delta$ of **0.6311 (All)** and **0.7304 (DEG)**, supporting its predictive performance in this setting. We will clarify that combination effects are modeled **implicitly through nonlinear response prediction**, without a pair-specific interaction module before pooling, and distinguish this predictive evidence from mechanistic validation of genetic interactions. **W5: Robustness to substantial batch shifts and generalization across biological contexts require more direct validation.** **1: Batch referencing and aggregation sensitivity.** BASIL defines perturbation responses using batch-matched controls: $$\Delta_{p,b}=\bar{y}_{p,b}-\bar{x}_b.$$ In the training ablation, batch matching increased Pearson $\Delta$ from **0.679 to 0.688** on RPE1 and from **0.654 to 0.671** on HepG2. We also replaced target-cell-count weights with uniform weights across all control batches, $w_b=1/B$, while keeping checkpoints, observed target populations, and scoring references fixed. Across **864 held-out perturbations**, the absolute changes in mean Pearson $\Delta$ were **0.0003, 0.0011, and 0.0022** on RPE1, HepG2, and Jurkat, respectively, indicating low sensitivity to this aggregation choice. These analyses support batch-matched referencing and aggregation stability; robustness to held-out batches or severe technical shifts remains to be evaluated directly. **2: Applicability across biological settings.** Our evaluation covers **CRISPRi perturbations in three cell lines, Norman CRISPRa combinations, and Combosciplex drug perturbations**. On seven held-out drug-treatment conditions, BASIL achieved Pearson $\Delta$ of **0.9369**, extending validation beyond genetic perturbations. Models were trained and evaluated separately for each dataset, supporting applicability across these settings. Cross-cell-line zero-shot transfer remains to be established. We will include the supplementary results and clarify these evaluation boundaries in the revised manuscript.Q1-Q3 Comment
Q1: Held-out predicted cell distributions need quantitative evaluation beyond mean-effect correlations. We evaluated single-cell predictions using , sliced Wasserstein distance (SWD), and variance retention. Within each perturbation–batch condition, we compared predicted and observed perturbed populations, using batch-matched unperturbed cells as the reference baseline. Populations were separately centered to remove mean shifts. Eligible groups contained at least eight control and eight target cells; we used matched sample sizes of up to 64 cells, three evaluation sampling repeats, and equal averaging across eligible perturbations. BASIL reduced centered across all three cell lines, as shown below. Centered SWD also decreased by 1.99%, 1.23%, and 0.73% on RPE1, HepG2, and Jurkat, respectively. Predicted-to-observed total-variance ratios were 0.859, 0.838, and 0.863, indicating substantial retention of cellular variation. These results support distributional fidelity within the evaluated held-out subsets after removing mean shifts. They compare the full model against matched controls and do not isolate MMD’s independent contribution. We will include the metrics, sampling protocol, and evaluation coverage in the revised manuscript and appendix.
| Dataset | Eligible/total held-out perturbations | Control centered ↓ | BASIL centered ↓ | Relative reduction |
|---|---|---|---|---|
| RPE1 | 43/387 | 0.02167 | 0.02070 | 4.48% |
| HepG2 | 16/207 | 0.02192 | 0.02138 | 2.45% |
| Jurkat | 50/270 | 0.02176 | 0.02133 | 1.99% |
Q2: Conditional MMD’s gains require uncertainty estimates across the four random seeds. We trained with four random seeds, but the original aggregation code retained only mean scores, so the tables did not include standard deviations or confidence intervals. We will report for the principal comparisons and ablations in the revised manuscript. For full BASIL versus the no-MMD variant, we will additionally report seed-wise paired performance differences and a confidence interval for the mean difference, assessing whether the gains exceed training variability.
Q3: Graph ablations are needed to quantify improvements beyond pretrained GenePT representations. We evaluated graph variants on 274 held-out perturbations in the Jurkat unseen-gene setting, keeping GenePT embeddings, training samples, and evaluation targets identical. The no-graph control preserves encoder layers, node-wise transformations, and the nominal parameter count, while setting all graph messages to zero. The full three-relation model improved clustering agreement by 2.40% over this control and by 7.91%, 3.55%, and 2.53% over TF-only, GO-only, and STRING-only controls, respectively. This metric compares the clustering structures of predicted and observed perturbation-mean expression profiles, supporting graph propagation’s contribution beyond GenePT. Relation-removal ablations provide complementary evidence: removing TF or STRING increased MSE from 0.004337 to 0.004413 and 0.004384, respectively; retaining these relations reduced MSE by 1.73% and 1.09%. We will include the complete no-graph, single-relation, and relation-removal configurations and results in the revised manuscript and appendix.
| Perturbation encoder | Clustering agreement |
|---|---|
| GenePT + node-wise encoder, without graph messages | 0.2821 |
| GenePT + TF messages only | 0.2677 |
| GenePT + GO messages only | 0.2789 |
| GenePT + STRING messages only | 0.2817 |
| GenePT + TF, GO, and STRING messages | 0.2888 |
Q4 Comment
Q4: BASIL’s performance needs comparison with an additive predictor across levels of non-additivity. We evaluated existing predictions on 37 held-out Norman combinations, with both constituent single-gene conditions available in training. The additive baseline uses only training single-gene responses, , with the same batch references and aggregation weights as BASIL. On the shared normalized log-expression scale, we defined empirical non-additivity as:
s_{AB}=\frac{\left\\|\delta_{AB}-(\delta_A+\delta_B)\right\\|_2}{\left\\|\delta_A\right\\|_2+\left\\|\delta_B\right\\|_2},
where is the observed combination response, and divided combinations into strength tertiles. In the strongest stratum, BASIL reduced mean MSE by 37.99%, improving 8/12 combinations. Pearson was comparable: 0.9172 for BASIL versus 0.9171 for the additive baseline. Using disjoint target-cell halves for stratification and evaluation, then swapping their roles, yielded strong-stratum MSE reductions of 34.64% and 29.84%. The benefit was specific to strongly non-additive combinations; the additive baseline performed better in the weaker strata. We report this as a descriptive improvement, since the combination-bootstrap confidence interval for the paired mean MSE benefit includes zero. We will include the complete stratified results and evaluation protocol in the revised manuscript.
| Non-additivity | Combinations | Additive MSE ↓ | BASIL MSE ↓ |
|---|---|---|---|
| Weak | 13 | 0.003224 | 0.005396 |
| Moderate | 12 | 0.002042 | 0.003747 |
| Strong | 12 | 0.007993 | 0.004956 |
W1-W2 Comment
W1: BASIL’s methodological contribution and the sources of its gains need clearer distinction from established approaches. 1: Shared response formulation and relation to prior work. BASIL’s contribution is a shared, state-dependent response operator jointly trained at population-mean and single-cell resolutions. For each perturbation–batch condition (p,b), mean supervision targets the observed response yˉp,b−xˉb, while conditional MMD constrains the population generated from individual control cells:
\mathcal{L}=\mathcal{L}_{\mathrm{mean}}+\lambda(t)\mathcal{L}_{\mathrm{MMD}}.$$ Both objectives update the **same response function**, without assuming correspondence between individual control and perturbed cells. Mean supervision anchors the population response, while conditional MMD constrains cell-level outputs beyond their mean. TxPert combines biological graphs and basal-state representations primarily through expression-MSE training; BASIL couples explicit mean-response supervision with conditional population alignment. scDFM combines conditional flow matching and MMD to learn a time-dependent velocity field; BASIL directly learns an expression-response mapping at both resolutions. We will make these distinctions explicit in the revision. **2: Controlled evidence and component attribution.** We retained GenePT, graph encoders, batch-matching rules, control pools, and conditional MMD, while replacing pseudobulk inputs and response targets with randomly paired single-cell inputs and targets. BASIL achieved higher Pearson $\Delta$ across all three cell lines, supporting the training design with representations and reference rules held fixed: | Cell line | BASIL | Single-cell pairing + MMD | BASIL gain | | --- | ---: | ---: | ---: | | RPE1 | **0.6882** | 0.6608 | **+0.0274** | | HepG2 | **0.6714** | 0.6234 | **+0.0480** | | Jurkat | **0.5389** | 0.5183 | **+0.0206** | In Norman’s two-unseen setting, removing mean-response supervision while retaining the priors, graph encoder, and MMD also reduced Pearson $\Delta$ from **0.6311 to 0.5201**, an absolute decrease of **0.1110**. Our attribution is therefore specific: **GenePT and mean-response supervision provide the main gains in mean prediction; batch matching defines the reference, and conditional MMD provides a complementary population constraint.** **W2: Conditional MMD’s modest gains in Pearson $\Delta$ require evidence beyond mean-level prediction and direct distributional evaluation.** **1: Matched ablation beyond Pearson $\Delta$.** Pearson $\Delta$ measures mean responses and does not directly assess cellular heterogeneity. In the matched full/no-MMD ablation, retaining conditional MMD improved clustering agreement across all three cell lines. This metric compares the clustering structures of predicted and observed **perturbation-mean expression profiles**, supporting a contribution to cross-perturbation response structure. MSE also decreased by **3.19%** on RPE1 and **0.95%** on Jurkat. | Dataset | Without conditional MMD | Full BASIL | Relative improvement | | --- | ---: | ---: | ---: | | RPE1 | 0.1308 | **0.1371** | **4.74%** | | HepG2 | 0.2495 | **0.2594** | **4.00%** | | Jurkat | 0.2405 | **0.2786** | **15.83%** | **2: Direct single-cell distributional evaluation.** We separately centered predicted and observed populations to remove mean shifts, then evaluated the full model against **batch-matched unperturbed controls**. Within eligible held-out perturbations, BASIL reduced centered $\mathrm{MMD}^2$ and centered sliced Wasserstein distance across all three datasets. Predicted-to-observed total-variance ratios of **0.838–0.863** also indicate substantial retained cellular variation. These results assess the full model’s distributional fidelity; the matched ablation above separately assesses conditional MMD’s contribution to response structure. We will include both analyses and their evaluation protocols in the revised manuscript. | Dataset | Eligible/total held-out perturbations | Centered $\mathrm{MMD}^2$ reduction vs. controls | Predicted/observed total variance | | --- | ---: | ---: | ---: | | RPE1 | 43/387 | **4.48%** | 0.859 | | HepG2 | 16/207 | **2.45%** | 0.838 | | Jurkat | 50/270 | **1.99%** | 0.863 |