acceptodds
Under review as a conference paper at ICLR 2027

PIE: Generalizing perturbation effects across unseen perturbations, contexts and datasets

Abstract

Predicting cellular response to perturbations is key to understanding biological mechanisms and selecting therapeutic targets. However, generalizing across cellular contexts, perturbations, and experimental datasets remains challenging due to the difficulty of measuring true perturbation effects and incomplete representations of the system being perturbed. We introduce PIE (Perturbation is Everything), which addresses these challenges by reformulating the learning task around population-level perturbation effects and incorporating auxiliary inputs that better characterize the biological system. PIE directly predicts differentially expressed genes and changes in gene expression for each (context, perturbation) pair, using external biological knowledge, baseline gene expression, and observed perturbation responses. On the Replogle-Nadig dataset, PIE achieves state-of-the-art performance on most metrics across all tested generalization settings, including the most challenging setting where both the context and perturbation are unseen during training. PIE achieves 1.2–3.2 times the AUPRC of the strongest baseline in each setting for predicting differentially expressed genes: across unseen contexts, unseen perturbations, and jointly unseen contexts and perturbations. PIE also outperforms existing baselines on zero-shot transfer across experimental datasets. Overall, PIE provides a framework for predicting cellular responses across datasets, contexts, and perturbations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Out-of-distribution generalization, perturbation response prediction, single-cell genomics, zero-shot transfer, multimodal learning, transformers, computational biology

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jjKqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PIE predicts population-level cellular perturbation effects rather than individual post-perturbation cells. It combines biological knowledge embeddings, control expression, and summaries of observed perturbation responses through a Perceiver IO architecture. Three output heads predict differential expression (DE), log fold changes, and discretized expression changes. The model supports unseen contexts, unseen perturbations, their combination, and cross-dataset transfer. However, the last comparison is against a zero-shift control baseline, making it substantially weaker.

Critical strengths
  1. The paper directly predicts population-level effects aligns training with common downstream analyses. This also avoids requiring individual-cell counterfactual targets that are not directly observed.

  2. The paper evaluates across contexts, perturbations, and datasets. Its joint generalization is particularly relevant to experimental prioritization.

  3. Training all subsets of input groups and computing exact Shapley values provides useful evidence about which signals matter. The authors also acknowledge results that weaken their narrative.

Critical weaknesses
  1. The headline AUPRC improvements do not clearly establish perturbation-specific generalization. This is my main concern. Section 4.4 reports that, when both entities are unseen, adding all five input groups improves AUPRC by only 0.010. Combined with the reported full-model result, the model without these groups achieves approximately 0.229 versus 0.239. Under the described architecture, that ablated model retains output-gene embeddings but receives no condition-specific biological inputs. Its performance therefore appears largely attributable to a learned, broadly shared ranking of genes likely to become DE. Similarly, for unseen perturbations, the total input-group contribution is only 0.045—much smaller than the headline improvement over competing methods.

  2. In the jointly unseen setting, comparing only against zero expression change is insufficient. Simple models can accept unseen context and perturbation embeddings—for example, ridge regression, a bilinear model, or an MLP trained on the same targets and inputs. TxPert (https://www.nature.com/articles/s41587-026-03113-4) is also a particularly relevant missing experimental baseline. It is cited in the submission and evaluates both unseen genetic perturbations and cross-cell-line transfer using biological knowledge graphs. Comparisons in compatible settings would strengthen the state-of-the-art claim.

  3. Population-level summary-statistic prediction already appears in Rhaister (https://www.biorxiv.org/content/10.64898/2026.06.09.731197v1.full.pdf), which directly models fold changes, significance statistics, and expression deltas. Thus, this formulation alone is not new. PIE’s contribution is better understood as combining that formulation with richer knowledge inputs and a flexible architecture. However, the current ablations primarily isolate input sources. They do not establish how much improvement requires.

  4. Some auxiliary inputs, including DepMap and L1000, contain independently measured perturbational information. This is legitimate, but differs from predicting a perturbation with no prior experimental characterization. Training time, parameter count, inference cost, and embedding-preparation costs would help assess whether the architecture offers practical advantages over simpler alternatives.

Questions for authors
  1. How much performance is perturbation-specific? Can you report gene-wise DE-frequency and gene-only baselines, together with perturbation-input shuffling? In particular, how do you interpret the approximately 0.229 AUPRC without the five input groups versus 0.239 with them when both entities are unseen?

  2. How robust are the gains to output conversion? Do the AUPRC improvements persist with a shared DE-calibration procedure for pseudobulk models, different synthetic-cell counts, and reduced p-value saturation?

11 comments
AuthorsOpenReview ↗

Question 1: How much performance is perturbation-specific

We appreciate the reviewer’s suggestion to probe the perturbation specific response prediction of PIE. For this investigation, we developed four new baselines. Across all but a few metrics, PIE consistently outperforms all baselines. Moreover, we perform an ablation where we shuffle the perturbation specific input, namely the knowledge sources and same perturbation evidence, across perturbations during inference. PIE’s performance in all generalization settings, including joint unseen setting, degrades when the perturbation specific input is shuffled. These results show that PIE’s predictions are perturbation specific, even in the unseen perturbation and joint unseen settings.

Population summary baselines and results:

We define the baselines as below:

  1. Population statistics, global: This is a parameter-free baseline. It outputs the DE-frequency, mean LFC and mean pseudobulk delta expression across all conditions in the train set. It is invariant to the context or perturbation identity.
  2. Population statistics, per context: This is the same population statistic baseline, however, with the statistics calculated over all perturbations of the query context only. It is context-aware but invariant to perturbation identity.
  3. Gene-text MLP: It is a 4.7M parameter MLP trained on NCBI gene text embeddings that are used as gene query embeddings in the decoder module of PIE. It is a trained model and is unaware of context or perturbation identity.
  4. Gene-only PIE: We train the PIE architecture, using the NCBI gene text embeddings as the gene query embeddings in PIE. However, we restrict the model by removing all other input components during training and evaluation. Thus this model is invariant to context and perturbation identity.

In the tables below, we compare these baselines and Context Mean (Adduri et al., Cell 2026) to PIE. These results obtained on the within dataset task on Replogle-Nadig (Replogle et al., Cell 2022; Nadig et al., Nature Genetics 2025), where we train each model on the train set, and evaluate on all three generalization settings.

Replogle-Nadig within dataset, unseen context, seen perturbation

ModelAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE0.424 ± 0.0340.185 ± 0.0230.819 ± 0.0260.734 ± 0.0520.664 ± 0.0460.764 ± 0.0610.313 ± 0.026
Context mean0.085 ± 0.0230.076 ± 0.0350.000 ± 0.000undefinedundefined0.502 ± 0.000undefined
Population statistics, global0.232 ± 0.0630.010 ± 0.0140.689 ± 0.031undefined0.400 ± 0.1230.502 ± 0.0010.263 ± 0.090
Gene-text MLP0.246 ± 0.0610.119 ± 0.0340.635 ± 0.051undefined0.315 ± 0.1030.503 ± 0.0010.222 ± 0.077
Gene-only PIE0.236 ± 0.0530.119 ± 0.0340.674 ± 0.037undefined0.346 ± 0.1160.503 ± 0.0010.219 ± 0.072

Replogle-Nadig within dataset, seen context, unseen perturbation

ModelAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE0.304 ± 0.0230.168 ± 0.0110.775 ± 0.0220.525 ± 0.0300.552 ± 0.0220.709 ± 0.0060.339 ± 0.023
Context mean0.082 ± 0.0090.063 ± 0.0110.675 ± 0.025undefined0.372 ± 0.0350.503 ± 0.0000.345 ± 0.031
Population statistics, global0.258 ± 0.0220.029 ± 0.0520.693 ± 0.015undefined0.411 ± 0.0510.503 ± 0.0000.308 ± 0.032
Population statistics, per context0.283 ± 0.0240.082 ± 0.0120.716 ± 0.014undefined0.455 ± 0.0430.503 ± 0.0000.352 ± 0.033
Gene-text MLP0.263 ± 0.0240.130 ± 0.0180.657 ± 0.040undefined0.369 ± 0.0460.503 ± 0.0000.260 ± 0.038
Gene-only PIE0.261 ± 0.0240.131 ± 0.0180.686 ± 0.013undefined0.390 ± 0.0520.503 ± 0.0000.259 ± 0.036
AuthorsOpenReview ↗

Question 1: How much performance is perturbation-specific

Continuing from previous comment on Question 1: How much performance is perturbation-specific

Replogle-Nadig within dataset, joint unseen

ModelAUPRCJaccard scoreDE direction matchSpearman Effect SizeSpearman LFCDiscrimination Score L1Pearson delta
PIE0.239 ± 0.0540.116 ± 0.0220.721 ± 0.0220.450 ± 0.0610.460 ± 0.0590.643 ± 0.0490.231 ± 0.031
Context mean0.088 ± 0.0180.091 ± 0.0410.000 ± 0.000undefinedundefined0.502 ± 0.000undefined
Population statistics, global0.230 ± 0.0600.011 ± 0.0150.668 ± 0.018undefined0.362 ± 0.1130.502 ± 0.0000.263 ± 0.089
Gene-text MLP0.240 ± 0.0550.116 ± 0.0340.641 ± 0.038undefined0.330 ± 0.0920.503 ± 0.0010.221 ± 0.075
Gene-only PIE0.233 ± 0.0500.117 ± 0.0340.672 ± 0.025undefined0.354 ± 0.1090.503 ± 0.0010.218 ± 0.071

PIE outperforms these population baselines on unseen context generalization. For unseen perturbation generalization, PIE outperforms baselines on all metrics except Pearson Delta. The joint unseen setting is the hardest. PIE is at par with these population baselines on AUPRC and Jaccard score, while outperforming on other DE-focused metrics like DE direction match and Spearman LFC. This shows that PIE is able to predict perturbation specific direction and magnitude for the joint unseen setting. PIE also outperforms these population baselines on Discrimination score L1, which requires perturbation specific predictions to perform better than random.

Shuffling perturbation specific inputs during inference: Starting with the trained PIE checkpoint on the within dataset task for Replogle-Nadig, we shuffle the knowledge sources, the same perturbation evidence or both at inference time. In the tables below, we compare these ablations to PIE (no shuffle) across all three generalization settings.

Replogle-Nadig within dataset, unseen context, seen perturbation

ModelAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (no shuffle)0.424 ± 0.0340.185 ± 0.0230.819 ± 0.0260.734 ± 0.0520.664 ± 0.0460.764 ± 0.0610.313 ± 0.026
Knowledge sources shuffled0.292 ± 0.0430.093 ± 0.0140.718 ± 0.0300.138 ± 0.0320.472 ± 0.0890.550 ± 0.0110.203 ± 0.052
Same-pert evidence shuffled0.258 ± 0.0510.131 ± 0.0170.772 ± 0.0290.689 ± 0.0540.550 ± 0.0440.716 ± 0.0680.269 ± 0.030
Both shuffled0.186 ± 0.0550.066 ± 0.0160.612 ± 0.0340.005 ± 0.0330.239 ± 0.1010.500 ± 0.0030.125 ± 0.041

Replogle-Nadig within dataset, seen context, unseen perturbation

ModelAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (no shuffle)0.304 ± 0.0230.168 ± 0.0110.775 ± 0.0220.525 ± 0.0300.552 ± 0.0220.709 ± 0.0060.339 ± 0.023
Knowledge sources shuffled0.213 ± 0.0130.093 ± 0.0060.646 ± 0.014−0.005 ± 0.0160.308 ± 0.0210.498 ± 0.0070.194 ± 0.022

Replogle-Nadig within dataset, joint unseen

ModelAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (no shuffle)0.239 ± 0.0540.116 ± 0.0220.721 ± 0.0220.450 ± 0.0610.460 ± 0.0590.643 ± 0.0490.231 ± 0.031
Knowledge sources shuffled0.191 ± 0.0550.072 ± 0.0220.630 ± 0.031−0.020 ± 0.0520.271 ± 0.0960.499 ± 0.0070.143 ± 0.048

Upon shuffling the perturbation-specific input components at inference, we observe that the model underperforms on all metrics across all three generalization settings. For all three generalization settings, removing all sources of perturbation signal reduces the Spearman effect size and Discrimination score L1 metrics to those equivalent to a random predictor.

These results suggest PIE is able to leverage its perturbation signal carrying inputs to predict perturbation-specific responses, even for unseen perturbations.

AuthorsOpenReview ↗

Question 1: How much performance is perturbation-specific

Continuing from previous comment on Question 1: How much performance is perturbation-specific

Interpreting the contribution of perturbation knowledge sources: We also acknowledge the reviewer observation regarding pointing out the small contribution of input components towards AUPRC in the joint unseen setting. During the revision process, we realized that our attribution analysis, as shown in Figure 4, was based on an older formulation of PIE. We have re-run our attribution analysis on the canonical version that is reflected in tables 3–5, and have updated Figure 4. Based on these attribution analyses, in the joint unseen setting, the gain in AUPRC between all input components on and off is +0.006.

As shown in the results above, the joint unseen setting is the hardest generalization setting. In this setting, the knowledge sources are the only perturbation-specific input to PIE. In this setting, PIE with knowledge sources available performs at par with population level baselines when predicting DE membership, as shown by AUPRC and Jaccard score metrics. Training without knowledge sources leads to a comparable checkpoint. However, in this generalization setting, PIE is able to leverage the knowledge sources to gain an advantage in predicting DE direction and magnitude, as well as predicting a discriminative pseudobulk delta expression. This is also reflected in the Shapley attribution scores in this setting, with knowledge sources having a negative contribution of −0.012 and −0.006 towards AUPRC and Jaccard score respectively, while having positive contributions of +0.060, +0.069 and +0.141 towards DE direction match, Spearman LFC and Discrimination Score L1 respectively. This further highlights that the different input components of PIE, including perturbation knowledge sources, vary in contribution across generalization settings and metrics.

Reviewer_p9fPOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PIE predicts gene-level DE calls, LFCs, and population-level mean expression changes for unseen context-perturbation pairs. It combines external knowledge, control expression, and response summaries, and is evaluated across unseen contexts, perturbations, datasets, and their combinations.

Critical strengths

PIE focuses on population-level perturbation effects that can be estimated directly from current screening data, and it systematically evaluates generalization across context, perturbation, and dataset axes. The experiments report complementary metrics for DE detection, response direction, effect size, and response discrimination.

Critical weaknesses

1.The pseudobulk formulation is insufficiently justified. It avoids unobserved cell-level counterfactuals but removes heterogeneity and subpopulation responses. Without a matched cell-level or distributional comparison using the same external inputs, it is unclear whether this choice improves generalization or simply lowers task resolution.

2.The source of the zero-shot gains is unclear. PIE uses external experimental profiles, including DepMap, L1000, PRISM, and JUMP, yet the ablations neither separate these signals from descriptive priors nor extend to cross-dataset or chemical settings. The gains may therefore reflect transfer from external screens rather than PIE's modeling design.

3.Evaluation focuses on aggregate metrics. The paper does not test whether top predicted genes, response directions, or pathways are biologically plausible on held-out conditions against strong baselines, weakening its claims about mechanistic interpretation and target discovery.

Questions for authors

1.Could the authors report results using only descriptive/structural priors, only external experimental profiles, and both together? This would be particularly useful for unseen-perturbation, joint-unseen, and cross-dataset settings.

2.Could the authors include simple matched baselines that use exactly the same external inputs, such as nearest-neighbor retrieval, linear regression, or a lightweight predictor? In the joint-unseen setting, comparison is currently limited to context mean, making the architectural contribution difficult to assess.

11 comments
AuthorsOpenReview ↗

Question 1: Descriptive versus experimental knowledge sources

We thank the reviewer for their suggestion for isolating the contributions of different types of knowledge sources towards PIE’s performance. We group the knowledge source set into two groups, Descriptive/Structural sources, that include natural language and structure annotations, and Experimental sources, that include external screens. We ablate these source groups on both the within dataset task and cross dataset task in the Replogle-Nadig dataset. The results show that while external screens typically contribute more towards metric performance as compared to Descriptive/Structural priors, they are not the exclusive drivers of performance. PIE with only Descriptive/Structural priors retains a comparable performance with respect to PIE with all knowledge sources.

Ablation of knowledge source groups:

We classify NCBI Gene Info and ESM2 embeddings as descriptive/structure based sources. We classify the NCBI Gene Annotations, STRING and DepMap to be experimental assay based sources. We included NCBI Gene Annotations in the later classification since it contains pathway association information from the GO (Ashburner et al., Nature Genetics 2000; The Gene Ontology Consortium et al., Nucleic Acids Research 2025) database. We present three arms, two that subset the knowledge sources respectively, while the third being the canonical arm that includes all 5 knowledge sources. Each arm is trained and evaluated with its subset of sources. While Experimental sources tend to have higher contributions in several metrics and generalization settings, PIE’s performance isn’t exclusively driven by this source class. The Descriptive/Structure sources provide meaningful contributions as well.

Note that we have updated our analysis in figure 7 to match the correct formulation of PIE in tables 3-7.

Replogle-Nadig, unseen context, seen perturbation

PIE armAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (all sources)0.424 ± 0.0340.185 ± 0.0230.819 ± 0.0260.734 ± 0.0520.664 ± 0.0460.764 ± 0.0610.313 ± 0.026
Descriptive/Structure (text + ESM2)0.412 ± 0.0310.185 ± 0.0200.821 ± 0.0310.752 ± 0.0380.652 ± 0.0570.770 ± 0.0640.319 ± 0.031
Experimental (STRING + DepMap + NCBI text)0.413 ± 0.0320.181 ± 0.0210.814 ± 0.0290.732 ± 0.0420.653 ± 0.0510.773 ± 0.0490.309 ± 0.028

Replogle-Nadig, unseen perturbation, seen context

PIE armAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (all sources)0.304 ± 0.0230.168 ± 0.0110.775 ± 0.0220.525 ± 0.0300.552 ± 0.0220.709 ± 0.0060.339 ± 0.023
Descriptive/Structure (text + ESM2)0.293 ± 0.0260.156 ± 0.0130.764 ± 0.0070.449 ± 0.0120.524 ± 0.0250.676 ± 0.0100.323 ± 0.014
Experimental (STRING + DepMap + NCBI text)0.315 ± 0.0250.173 ± 0.0120.783 ± 0.0170.608 ± 0.0290.563 ± 0.0330.738 ± 0.0080.352 ± 0.024

Replogle-Nadig, joint unseen

PIE armAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (all sources)0.239 ± 0.0540.116 ± 0.0220.721 ± 0.0220.450 ± 0.0610.460 ± 0.0590.643 ± 0.0490.231 ± 0.031
Descriptive/Structure (text + ESM2)0.228 ± 0.0390.113 ± 0.0150.713 ± 0.0120.403 ± 0.0540.426 ± 0.0920.627 ± 0.0390.230 ± 0.046
Experimental (STRING + DepMap + NCBI text)0.242 ± 0.0380.122 ± 0.0220.700 ± 0.0310.502 ± 0.1020.447 ± 0.0660.671 ± 0.0430.235 ± 0.039
AuthorsOpenReview ↗

Question 1: Descriptive versus experimental knowledge sources

Continuing from previous comment on Question 1: Descriptive versus experimental knowledge sources

The Shapley values of each source class can be calculated as the sum of its member sources. We provide these per source class Shapley values in the table below.

Replogle-Nadig, unseen context, seen perturbation

Source classAUPRCJaccard scoreDE direction matchSpearman LFCDiscrimination Score L1Pearson delta
Descriptive (text + ESM2)+0.005 ± 0.006+0.014 ± 0.011+0.007 ± 0.009+0.017 ± 0.025+0.019 ± 0.004+0.000 ± 0.011
Experimental (STRING + DepMap + NCBI text)+0.012 ± 0.010+0.025 ± 0.023+0.004 ± 0.018+0.023 ± 0.022+0.033 ± 0.020−0.011 ± 0.028

Replogle-Nadig, unseen perturbation, seen context

Source classAUPRCJaccard scoreDE direction matchSpearman LFCDiscrimination Score L1Pearson delta
Descriptive (text + ESM2)−0.002 ± 0.001−0.003 ± 0.002+0.018 ± 0.015+0.027 ± 0.020+0.047 ± 0.010+0.007 ± 0.009
Experimental (STRING + DepMap + NCBI text)+0.025 ± 0.003+0.029 ± 0.008+0.058 ± 0.020+0.083 ± 0.022+0.158 ± 0.010+0.042 ± 0.010

Replogle-Nadig, joint unseen

Source classAUPRCJaccard scoreDE direction matchSpearman LFCDiscrimination Score L1Pearson delta
Descriptive (text + ESM2)−0.009 ± 0.009−0.013 ± 0.012+0.023 ± 0.013+0.023 ± 0.026+0.033 ± 0.010−0.002 ± 0.017
Experimental (STRING + DepMap + NCBI text)+0.003 ± 0.009+0.004 ± 0.017+0.052 ± 0.032+0.072 ± 0.058+0.107 ± 0.040+0.003 ± 0.041

Furthermore, we ran the same arms in the cross dataset setting in Replogle-Nadig. For all genetic perturbation datasets used in this setting, we used the same knowledge sources, classified into Descriptive/Structure or Experimental sources in the same manner. Since Tahoe-100M (Zhang et al., bioRxiv 2025) is also included as a training dataset in this recipe, we made a similar classification for its knowledge sources. We classified PubChem annotations and SMILES embeddings as Descriptive/Structure sources, and L1000, PRISM and JUMP as Experimental sources. The results for the cross dataset analysis are presented in the table below.

Replogle-Nadig cross dataset, seen perturbations

PIE armAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (all sources)0.378 ± 0.0330.138 ± 0.0090.727 ± 0.0350.627 ± 0.0640.508 ± 0.0430.620 ± 0.0550.136 ± 0.022
Descriptive/Structure (text + ESM2)0.368 ± 0.0250.141 ± 0.0100.714 ± 0.0370.579 ± 0.0430.496 ± 0.0340.624 ± 0.0460.123 ± 0.016
Experimental (STRING + DepMap + NCBI text)0.374 ± 0.0290.156 ± 0.0130.742 ± 0.0290.634 ± 0.0720.519 ± 0.0530.635 ± 0.0480.137 ± 0.028

Replogle-Nadig cross dataset, unseen perturbations

PIE armAUPRCJaccard scoreDE direction matchSpearman effect sizeSpearman LFCDiscrimination Score L1Pearson delta
PIE (all sources)0.252 ± 0.0610.123 ± 0.0180.603 ± 0.0340.560 ± 0.0290.278 ± 0.0890.515 ± 0.0090.067 ± 0.033
Descriptive/Structure (text + ESM2)0.244 ± 0.0530.108 ± 0.0100.574 ± 0.0330.365 ± 0.0210.185 ± 0.0080.514 ± 0.008−0.016 ± 0.025
Experimental (STRING + DepMap + NCBI text)0.248 ± 0.0590.128 ± 0.0120.638 ± 0.0160.511 ± 0.0170.282 ± 0.0610.543 ± 0.0220.075 ± 0.032

The results show that contributions from source classes vary by generalization setting and metric.

AuthorsOpenReview ↗

Question 2: Simple input-matched baselines

While we agree with the reviewer that clearly highlighting the benefits of architecture choice is important for this work, it is also relevant to note that architecture choice is only one component of the broader modeling framework introduced by PIE. Other innovations include dataset selection and featurization, data representation, training task selection, multi-head prediction and the inclusion of gene summary statistics. Together, these choices aim to identify and leverage reliable perturbation signals to improve generalization across diverse settings. We chose the Perceiver architecture to optimize for flexibility, with the goal of understanding the contributions of different data sources and training signals across generalization settings, as well as enabling rapid adaptation to new data sources and feature types.

To isolate the contribution of architecture choice, we followed the reviewer’s recommendation and performed an architecture ablation by training three simplified models that use the same inputs as PIE and predict in the same output space. Across all generalization settings in the within-dataset task, PIE outperforms simplified baselines such as nearest neighbor and ridge regression on most metrics, while achieving comparable performance to an MLP. These results demonstrate that PIE’s predictive performance is not simply attributable to the use of a more expressive architecture, but also reflects the broader modeling choices underlying our approach.

Reviewer_xrawOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces PIE a framework for predicting perturbation responses across unseen cellular contexts, perturbations, and datasets. The main idea is to simplify perturbation prediction by operating at the population level instead of predicting individual perturbed cells. PIE combines various prior knowledge including baseline expression from control cell lines, external representations of contexts and perturbations, and summary statistics of responses observed in other training conditions. The architecture is based on a Perceiver-style encode-process-decode framework, where knowledge-source tokens are encoded into latent representations and individual genes query these representations to generate predictions.

Critical strengths
  • The paper shows that the model could operate on an entirely new dataset which is a novel task compare to prior works.
  • The model works on both molecular and genetic perturbation prediction problem.
Critical weaknesses
  • My main concern is that the paper does not directly evaluate how accurate PIE predicts quantitative perturbation responses i.e. the reported metrics do not include a direct correlation or error between predicted and observed responses. This is a major concern especially for the Tahoe dataset since the improvement in DE prediction can be large while the Discrimination Score remains close to random in some generalization settings.
  • I would also like to better understand how much performance comes from external experimental measurements. For example, unseen chemical perturbations can still have response-related information from L1000, PRISM, and JUMP.
  • The baseline comparison is relatively weak when both context and perturbation are unseen
Questions for authors
  • PIE's "unseen perturbation" setting isn't always the same as predicting a completely unknown drug from its chemical structure alone. Since a drug might not appear in the training dataset but PIE still receives it's prior information for example Drug X is held out from Tahoe but it might still sees its gene expression response from L1000 dataset. How do you address such cases?
  • Why do the authors not report direct Pearson/Spearman correlation or error between predicted and observed pseudobulk delta-expression profiles, given that delta expression is explicitly predicted by the model?
  • For unseen contexts, how much improvement does PIE get over predicting mean response of the same perturbation across the training contexts?
  • For chemical perturbations, how does performance change when L1000, PRISM, and JUMP are removed?
9 comments
AuthorsOpenReview ↗

Question 1: Unseen drugs with external experimental profiles

We thank the reviewer for the question. To capture model performance in the setting of a completely unknown drug, we tested the effect of removing all knowledge sources based on experimental assays, such as L1000 (Subramanian et al., Cell 2017), PRISM (Corsello et al., Nature Cancer 2020), and JUMP (Chandrasekaran et al., bioRxiv 2023). Through these ablations we observe that restricting PIE to not use experimental priors from external assays, specifically L1000, PRISM and JUMP, does not cause a significant degradation in performance on metrics.

Ablation of priors from external assays: We train an arm of PIE that subsets the knowledge sources to descriptive and structure based ones, PubChem text description and SMILES. This arm did not see data from L1000, PRISM and JUMP during training or inference. In the table below, we compare this arm against PIE trained with all five knowledge sources.

Tahoe-100M, within dataset, unseen context seen perturbation:

AblationsAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
PIE0.680 ± 0.0940.400 ± 0.1120.872 ± 0.0530.759 ± 0.0900.721 ± 0.1040.982 ± 0.0130.691 ± 0.053
PIE − L1000, PRISM, JUMP0.683 ± 0.0930.402 ± 0.1080.874 ± 0.0500.767 ± 0.0740.723 ± 0.0980.983 ± 0.0110.693 ± 0.047

Tahoe-100M, within dataset, seen context unseen perturbation:

AblationsAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
PIE0.528 ± 0.1090.316 ± 0.0800.682 ± 0.0940.031 ± 0.0820.454 ± 0.1590.531 ± 0.0100.239 ± 0.050
PIE − L1000, PRISM, JUMP0.530 ± 0.1080.318 ± 0.0790.690 ± 0.0910.029 ± 0.0780.459 ± 0.1570.544 ± 0.0120.263 ± 0.052

Tahoe-100M, within dataset, joint unseen:

AblationsAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
PIE0.536 ± 0.1050.288 ± 0.0820.612 ± 0.024−0.005 ± 0.0660.323 ± 0.0570.523 ± 0.0060.189 ± 0.025
PIE − L1000, PRISM, JUMP0.540 ± 0.1030.287 ± 0.0830.625 ± 0.028−0.010 ± 0.0570.332 ± 0.0550.544 ± 0.0040.218 ± 0.026

Removing L1000, PRISM and JUMP lowers performance on certain metrics while improving on others.

To be more rigorous with our analysis, we ran ablations to measure the attribution of different input components of PIE to its performance in Tahoe-100M. We discovered that same-perturbation evidence was the leading driver of the model’s performance. This could be explained by high correlation in perturbation response across cell lines for drugs in Tahoe-100M. To remove this confounder, we trained the same two arms as above, but with the same-perturbation evidence disabled during training and inference. We provide the results across all three generalization settings below.

Tahoe-100M without same-perturbation evidence, within dataset, unseen context seen perturbation:

ModelAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
PIE0.647 ± 0.0960.381 ± 0.1120.845 ± 0.0590.759 ± 0.0710.667 ± 0.1230.969 ± 0.0170.643 ± 0.050
PIE − L1000, PRISM, JUMP0.652 ± 0.1000.390 ± 0.1010.849 ± 0.0520.747 ± 0.0500.674 ± 0.1030.962 ± 0.0210.644 ± 0.050

Tahoe-100M without same-perturbation evidence, within dataset, unseen perturbation seen context:

ModelAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
PIE0.526 ± 0.1100.307 ± 0.0750.688 ± 0.0910.037 ± 0.0850.463 ± 0.1540.542 ± 0.0100.257 ± 0.052
PIE − L1000, PRISM, JUMP0.521 ± 0.1090.308 ± 0.0760.679 ± 0.0940.074 ± 0.0850.443 ± 0.1620.535 ± 0.0100.248 ± 0.052

Tahoe-100M without same-perturbation evidence, within dataset, joint unseen:

ModelAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
PIE0.532 ± 0.1010.268 ± 0.0880.619 ± 0.025−0.002 ± 0.0390.337 ± 0.0620.533 ± 0.0060.209 ± 0.025
PIE − L1000, PRISM, JUMP0.530 ± 0.1050.281 ± 0.0800.615 ± 0.0270.052 ± 0.0280.323 ± 0.0590.530 ± 0.0040.202 ± 0.029

These results further show that removing L1000, PRISM and JUMP do not significantly degrade PIE’s performance on the metrics.

AuthorsOpenReview ↗

Question 2: Correlation with pseudobulk delta expression

We appreciate the reviewer’s suggestion to report additional metrics that focus on pseudobulk delta expression. We report the Pearson Delta metric (Adduri et al., Cell 2026) below, in addition to the Discrimination Score L1 metric.

Definition of Pearson Delta metric: Pearson delta is the Pearson correlation between the predicted and reference pseudobulk delta expression, Δ^g\hat\Delta_g and Δg\Delta_g, over all GG measured genes. Like the other per-condition metrics, it is computed per experimental condition and averaged over the perturbations of a context.

Metric comparison: The pseudobulk delta expression focused metric values for PIE in each generalization setting on the within dataset task in Replogle-Nadig (Replogle et al., Cell 2022; Nadig et al., Nature Genetics 2025) and Tahoe-100M (Zhang et al., bioRxiv 2025) are provided in the table below. For each table, we compare PIE to the strongest published baseline and Context Mean (Adduri et al., Cell 2026) in that setting.

Replogle-Nadig:

Generalization SettingModelDiscrimination Score L1Pearson delta
Unseen contextPIE0.764 ± 0.0610.313 ± 0.026
Unseen contextRhaister-O (Svensson et al., bioRxiv 2026)0.712 ± 0.0720.322 ± 0.027
Unseen contextContext Mean0.502 ± 0.000undefined
Unseen perturbationPIE0.709 ± 0.0060.339 ± 0.023
Unseen perturbationTabICL (Palla et al., bioRxiv 2026)0.682 ± 0.0100.441 ± 0.023
Unseen perturbationContext Mean0.503 ± 0.0000.345 ± 0.031
Unseen context + perturbationPIE0.643 ± 0.0490.231 ± 0.031
Unseen context + perturbationTxPert cross cell line (Wenkel et al., Nature Biotechnology 2026)0.534 ± 0.0210.279 ± 0.036
Unseen context + perturbationContext Mean0.502 ± 0.000undefined

Tahoe-100M:

Generalization SettingModelDiscrimination Score L1Pearson delta
Unseen contextPIE0.982 ± 0.0130.691 ± 0.053
Unseen contextRhaister-O0.970 ± 0.0190.730 ± 0.067
Unseen contextContext Mean0.501 ± 0.000undefined
Unseen perturbationPIE0.531 ± 0.0100.239 ± 0.050
Unseen perturbationContext Mean0.502 ± 0.0000.379 ± 0.058
Unseen context + perturbationPIE0.523 ± 0.0060.189 ± 0.025
Unseen context + perturbationContext mean0.502 ± 0.000undefined

Choosing a set of complementary and robust metrics: Our choice of evaluation metrics was motivated by two considerations: the complementary value each metric adds to the evaluation and the consistency and reliability of the experimental signals they capture across datasets. Single-cell perturbation screens are most widely used to identify significantly differentially expressed genes, characterize the direction and magnitude of perturbation effects on those genes, and cluster perturbations based on their effects. Measuring average changes in expression across all genes can be confounded by various sources of noise in single-cell perturbation datasets, and the reproducibility of this signal has not been demonstrated to be strong. Thus, when evaluating expression counts, we focused on Discrimination Score L1. This metric emphasizes the core perturbation-specific signal rather than broadly measuring expression changes across all genes.

For example, in the setting of seen context and unseen perturbation, Context Mean outperforms PIE in the unseen perturbation setting on Pearson Delta in both Replogle-Nadig and Tahoe-100M datasets. However, Context Mean is an average of responses across the query cell line and is not perturbation aware, leading to a Discrimination Score at par with random on both datasets.

AuthorsOpenReview ↗

Question 3: Gains over Perturbation Mean in unseen contexts

Tables 3–7 in the manuscript include the published baseline Perturbation Mean (Adduri et al., Cell 2026), which is an applied mean of pseudobulk delta expression across conditions in the train set that share the same perturbation as the query. This baseline seems the most aligned to the reviewer’s suggestion. PIE outperforms Perturbation Mean baseline on most metrics across datasets and generalization tasks.

Results comparing Perturbation Mean baseline: In the tables below, we have provided the metric comparison of PIE against Perturbation Mean across both the within dataset and cross dataset tasks on Replogle-Nadig (Replogle et al., Cell 2022; Nadig et al., Nature Genetics 2025), and the within dataset task on Tahoe-100M (Zhang et al., bioRxiv 2025) datasets.

Replogle-Nadig, within dataset, unseen context seen perturbation:

ModelAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
Perturbation Mean0.088 ± 0.0220.079 ± 0.0180.806 ± 0.0310.440 ± 0.1060.534 ± 0.0290.659 ± 0.0730.330 ± 0.017
PIE0.424 ± 0.0340.185 ± 0.0230.819 ± 0.0260.734 ± 0.0520.664 ± 0.0460.764 ± 0.0610.313 ± 0.026

Replogle-Nadig, cross dataset, seen perturbation:

ModelAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
Perturbation Mean0.062 ± 0.0130.052 ± 0.0100.733 ± 0.0340.439 ± 0.0770.472 ± 0.0410.561 ± 0.0260.212 ± 0.014
PIE0.378 ± 0.0330.138 ± 0.0090.727 ± 0.0350.627 ± 0.0640.508 ± 0.0430.620 ± 0.0550.136 ± 0.022

Tahoe-100M, within dataset, unseen context seen perturbation:

ModelAUPRCJaccardDE Direction MatchSpearman effect sizeSpearman LFCDiscrimination score L1Pearson delta
Perturbation Mean0.367 ± 0.1360.264 ± 0.0740.744 ± 0.0210.307 ± 0.0680.478 ± 0.0580.813 ± 0.0480.691 ± 0.071
PIE0.680 ± 0.0940.400 ± 0.1120.872 ± 0.0530.759 ± 0.0900.721 ± 0.1040.982 ± 0.0130.691 ± 0.053

PIE outperforms Perturbation Mean across all metrics focused on Differential Expression membership, magnitude and direction. Among the metrics focused on pseudobulk delta expression, PIE outperforms on Discrimination Score L1, while Perturbation Mean outperforms on Pearson Delta.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Summary

We thank the reviewers for their careful reading and constructive feedback. Their comments led to new experiments that test where PIE's gains come from, and to clearer reporting of PIE's quantitative accuracy.

Improvements in revised submission

In response to reviewer feedback:

Baselines: We added TxPert (three variants), four population-level baselines (global and per-context population statistics, a gene-text MLP and a gene-only PIE), and three input-matched architecture baselines (1-NN, ridge regression and a pooled MLP) that take exactly PIE's inputs and predict in its output space.

Knowledge source ablations: We trained PIE with only descriptive/structural sources and with only experimental sources on Replogle-Nadig, within and across datasets. We also trained PIE without L1000, PRISM and JUMP on Tahoe-100M, with and without same-perturbation evidence.

Perturbation specificity: We shuffled PIE's perturbation-specific inputs across perturbations at inference, in all three generalization settings.

Formulation: We compared a restricted PIE that receives only a one-hot perturbation and the control mean against State, which takes similar inputs and predicts cell-level responses.

Baseline scoring: We rescored the affected baselines with an unsaturated p-value conversion and with 250 to all control cells.

Metrics and figures: We report Pearson delta alongside Discrimination Score L1, report the compute cost of PIE and the baselines, and updated Figures 4 and 7 to the canonical PIE model used in Tables 3–7.

Contributions in our work

Population-level formulation: PIE predicts differentially expressed genes, log fold changes and pseudobulk delta expression for each (context, perturbation) pair, the quantities perturbation screens are most often used to estimate. With only a one-hot perturbation and the control mean as input, PIE reaches an AUPRC of 0.349 against 0.138 for State on unseen contexts.

Generalization across contexts, perturbations and datasets: PIE represents contexts and perturbations through external knowledge, control expression and observed responses. One model therefore covers unseen contexts, unseen perturbations, jointly unseen pairs and transfer across datasets. Population-level models such as Rhaister-O cannot natively address unseen perturbations or jointly unseen pairs.

State-of-the-art performance: On Replogle-Nadig, PIE achieves 1.2–3.2 times the AUPRC of the strongest baseline in each setting, and outperforms TxPert on all metrics except Pearson delta in every setting.

Perturbation-specific predictions: Shuffling the knowledge sources across perturbations lowers Discrimination Score L1 to random (0.499 against 0.643 for jointly unseen pairs), so PIE's predictions depend on the identity of the perturbation even when it was never observed in training.

Not reliant on external screens: With only descriptive and structural sources, PIE retains most of its performance (AUPRC 0.293 against 0.304 for unseen perturbations). On Tahoe-100M, removing L1000, PRISM and JUMP leaves performance comparable in all three settings.

Attribution of inputs: Exact Shapley values over every subset of PIE's input groups show that the contribution of each input differs by generalization setting and metric.

Practical cost: PIE has 79.1M parameters and trains in 2.2 GPU hours per fold, a quarter of the GPU hours of State.

Implications

Our results show that population-level effects carry the most reliable perturbation signal in current screens, and that external knowledge lets a model carry this signal to contexts and perturbations it has never observed. The input-matched baselines also show that, for some settings and metrics, a simpler trunk over the same inputs approaches PIE's performance. We chose the Perceiver trunk because it accepts variable-length, sparse knowledge sources and variable gene sets at a fixed parameter count, which lets new sources and datasets be added without changing the model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.