acceptodds
Under review as a conference paper at ICLR 2027

Learning Faithful Mechanism Subgraphs from Partially Trusted Biological Priors for Genetic Perturbation Prediction

Abstract

Predicting cellular responses to genetic perturbations is a central problem in virtual cell modeling. However, current methods face two key limitations: they typically treat biological priors as fixed ground truth, which are inherently incomplete, noisy, and only partially trustworthy; and they lack explicit mechanistic explanations for their predictions, instead relying on post hoc analyses rather than explicit, testable mediator entities. As a result, it remains unclear whether a model relies on biological mechanisms or merely exploits statistical correlations. These limitations motivate a broader machine learning question: how can we learn from partially trusted prior knowledge while ensuring that the learned explanation faithfully reflects the mechanism underlying the prediction? To address this question in genetic perturbation modeling, we propose **PerturbDigger**, a graph learning framework for perturbation prediction and mechanistic explanation. PerturbDigger formulates biological prior knowledge as a partially trusted prior graph, performs context-specific structural calibration to improve graph reliability, and learns perturbation responses with a biologically-guided graph transformer regularized by faithfulness constraints on mechanistic subgraphs. Extensive experiments show that PerturbDigger improves genetic perturbation prediction, generalizes better to unseen cellular contexts, and produces mechanistic subgraphs at both the sample and perturbation levels. Additional analyses show that the learned subgraphs are biologically meaningful, faithful, and stable, suggesting a promising step toward more interpretable and mechanism-aware virtual cell models.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Virtual cells, Genetic perturbation prediction, Mechanism subgraphs

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KgBaOpenReview ↗
Rating
2
Confidence
1
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PerturbDigger, a graph learning framework designed for genetic perturbation prediction and mechanistic explanation. To address the limitations of treating biological priors as fixed ground truth and lacking explicit mechanistic reasoning, PerturbDigger formulates biological prior knowledge as a partially trusted prior graph, performs context-specific structural calibration using control data, and predicts perturbation responses through a biologically-guided graph transformer regularized by faithfulness constraints on mechanistic subgraphs. Experiments across multiple benchmark datasets and a real-world cell context generalization benchmark demonstrate improved predictive performance and stable, faithful, and biologically meaningful mechanistic explanations.

Critical strengths
  • The formulation of biological priors as partially trusted graphs combined with context-specific structural calibration effectively bridges the gap between rigid prior structures and context-dependent regulatory relationships.
  • The biologically-guided graph transformer enforces ordered propagation through explicit biological mediators (such as TFs and pathways), offering intrinsic interpretability rather than relying solely on post hoc attributions.
  • The incorporation of faithfulness constraints (via counterfactual necessity) ensures that the learned mechanistic subgraphs are genuinely responsible for model predictions.
Critical weaknesses
  • The methodological framework heavily relies on the quality and coverage of external biological knowledge bases (such as CellOracle and Reactome). For poorly annotated genes or non-model organisms, the reliability of the partially trusted prior graph and calibration steps may degrade significantly.
  • While the paper emphasizes faithfulness via counterfactual necessity, the evaluation of the biological mechanisms heavily relies on qualitative case studies (e.g., literature-consistent subgraphs for CTNNB1 and EP300), lacking systematic quantitative validation against independent large-scale wet-lab perturbation benchmarks.
  • The computational overhead introduced by the multi-stage structural calibration and relation-specific constraints is insufficiently discussed regarding its scalability to genome-scale networks with hundreds of thousands of cells and diverse cell lines.
Questions for authors
  • How sensitive is PerturbDigger's performance to the choice and noise level of initial biological priors (e.g., missing edges in CellOracle or Reactome)? Can the calibration stage successfully recover heavily corrupted or entirely missing regulatory pathways?
  • In Section 3.3, the necessity loss relies on counterfactual masking of mediator edges. Since masking edges in a graph transformer can lead to distribution shifts, how does the model disentangle true mechanistic necessity from artifacts caused by breaking message-passing connectivity?
  • Could the authors clarify how the pruning budget parameter KK is tuned across datasets of varying sizes, and whether a fixed KK leads to structural bias when explaining perturbations that involve vastly different numbers of downstream target genes?
7 comments
AuthorsOpenReview ↗

Response to W1 & Q1

Dear Reviewer KgBa,

Thanks for your constructive comments. Below we will address your concerns.


W1 & Q1. Dependence on biological priors and robustness to incomplete or corrupted knowledge.

Thank you for highlighting this limitation. We agree that PerturbDigger depends on the coverage and quality of its candidate biological graph. However, our focus is to explain how perturbation effects propagate through biologically testable mediators, specifically, how biological priors can be calibrated to a context-specific cellular environment and used to derive an explanation of how the perturbation occurs, rather than to discover entirely new pathways from perturbation data. Consequently, if an edge is entirely absent from the candidate set, that exact edge cannot appear in the resulting mechanistic subgraph. We will explicitly clarify this scope and limitation in the revised Limitations section.

To evaluate the robustness to incomplete or corrupted priors, We further evaluate these two cases below.

(1) Robustness to incomplete priors.

We randomly removed 10%, 20%, and 30% of the TF–gene edges before calibration and retrained the complete pipeline. We evaluated predictive performance and external ENCODE support using the same evaluation protocol (details shown in Q2).

Robustness to incomplete TF–gene priors on Adamson. | Removed TF–gene edges | RMSE ↓ | Median AUPR ratio ↑ | |:---:|:---:|:---:| | 0% | 0.067 | 2.758 | | 10% | 0.069 | 2.442 | | 20% | 0.074 | 2.067 | | 30% | 0.083 | 1.769 |

Performance degrades progressively as more prior edges are removed. Nevertheless, the remaining relationships retain greater-than-chance ENCODE support even after 30% removal. These results demonstrate tolerance to moderate prior incompleteness while confirming that adequate candidate coverage remains important.

(2) Recovery from corrupted edge-reliability assignments.

We further examined whether recalibration can restore useful reliability assignments when the candidate relationships remain available but their weights are corrupted. Starting from the calibrated graph, we constructed two variants while keeping the candidate topology and trained perturbation predictor fixed:

  • A: Random weight corruption. We perturbed the calibrated edge weights with random weights under the [0,1][0,1] constraint.
  • B: High/low weight reassignment. We divided the edges into the upper and lower halves of the reliability ranking, independently shuffled the weights within each half, and exchanged the assignments between the two halves. This substantially disrupts which relationships receive high reliability while preserving the overall collection of weight values.

For A-re and B-re, we reran Stages I and II using the same control data, with Stage II initialized from the corresponding corrupted weights. The clean edge weights were not restored, and the perturbation predictor remained fixed.

Recovery of prediction performance after edge-weight corruption on Adamson. | Variant | RMSE ↓ | |:---|:---:| | Clean calibration | 0.067 | | A: Random weight corruption | 0.109 | | A-re: Recalibration after A | 0.068 | | B: High/low weight reassignment | 0.194 | | B-re: Recalibration after B | 0.073 |

Both corruption schemes substantially degrade prediction performance, whereas recalibration restores most of the lost performance. These results support the ability of control-based calibration to recover useful reliability assignments within the existing candidate topology.

(3) Contribution of our recovered edges to interpretation.

We additionally evaluated the explanation-verbalization stage on 50 held-out cases, using three experts and an LLM judge. For each case, we compared LLM-generated interpretations of the extracted mechanistic subgraph with interpretations generated after removing all edge relations while retaining the same nodes. The evaluators assessed the paired interpretations in a blinded comparison.

Blinded preference rates for LLM-generated interpretations based on the original mechanistic subgraphs versus the same nodes without edge relations. | |Original subgraph |Removing edges| |:---|:---|:--- | Expert 1|88%|12% | Expert 2|84%|16% | Expert 3|90%|10% | LLM-as-judge (GPT 6 Astra xhigh)|92%|8%

Interpretations based on the original subgraphs were preferred in 84%–92% of cases across evaluators. The experts observed more unsupported or biologically implausible regulatory and causal statements in the node-only condition. This sanity check indicates that the selected relational structure carries essential mechanistic information and helps constrain the resulting biological interpretation.

AuthorsOpenReview ↗

Response to W2 & Q2

W2. Systematic quantitative validation using independent experimental data.

Thank you for this suggestion. We complement the qualitative examples with systematic evaluations using independent binding and perturbation data.

TF–gene binding evidence. Following the ChIP-seq-based GRN evaluation protocol of LINGER [2], we evaluated PerturbDigger’s TF–gene edges against independent binding evidence from ENCODE. For each TF with ChIP-seq coverage, candidate edges were ranked by their mean gating scores across Adamson test samples, with pruned edges assigned zero. We report the AUPR ratio (AUPR normalized by the prevalence of ChIP-supported edges), where 11 represents random ranking and values above 11 indicate enrichment of experimentally supported binding edges.

ENCODE support for TF–gene edge rankings on Adamson. Variant|median AUPR ratio(↑) :-|:-: Random|1.000 Full model (Ours)|2.758 w/o calibration|1.842

PerturbDigger achieved a median AUPR ratio of 2.758, substantially exceeding both random ranking and the model without calibration. This provides independent binding evidence for the mediator subgraphs.

Direct mediator-perturbation evidence. We used the distinct Replogle K562 genome-wide CRISPRi screen, which targets all expressed genes and provides their observed transcriptional effects, as an external reference not used in model training or calibration. For each Adamson test perturbation pp, we extracted its selected gene mediator mm and compared the original perturbation effect with the independently measured CRISPRi effect of mm over the mediator’s selected downstream genes. We evaluated affected-gene overlap rather than numerical agreement. We report downstream-overlap enrichment (OE), defined as the observed overlap divided by its random expectation.

Functional support for selected mediators from independent perturbation data. ||mean OE(↑) :-|:-: Random|1.000 Ours|3.659

The greater-than-random enrichment provides independent functional support that the selected mediators affect the expected downstream genes.


Q2. Distinguishing predictive dependence from artifacts of graph disruption.

Thank you for raising this concern. PerturbDigger constrains propagation to a calibrated, biologically grounded mediator graph whose selected edges directly participate in forward computation. Within our model, dependence on these mediators is therefore implemented through their message-passing contributions. This design couples biological priors with model decisions, promoting both model faithfulness and alignment with biological evidence.

To address your concerns, we introduce two additional controls to examine whether the identified relationships contribute beyond the effects of disrupting connectivity. Both experiments use Adamson with a fixed intervention budget of 30% of the originally active mediator edges:

  • Structurally matched deletion. We compare top-ranked edge deletion with random deletion from the originally active mediator edges, matching the number of edges removed for each relation type at each receiving node. This controls both the amount and location of input loss, allowing us to assess whether top-ranked relationships contribute more than structurally matched alternatives.
  • Attenuation without removing connections. Using the same edge sets, we halve their gated message contributions instead of deleting them. This preserves the selected connections while reducing their influence, testing whether the observed dependence persists without complete disconnection.

Additional controls on Adamson. Entries report RMSE increases (ΔRMSE\Delta\mathrm{RMSE}) relative to the original prediction. Both interventions affect 30% of the active mediator edges.** | Intervention | Top-ranked edges | Matched-random edges | |:---|:---:|:---:| | Edge deletion | 0.021 | 0.006 | | Message attenuation by 50% | 0.015 | 0.003 |

Top-ranked interventions produce larger RMSE increases than the structurally matched random controls under both deletion and message attenuation. The persistence of this difference when connections are retained supports predictive dependence on the selected relationships and indicates that the observed effects cannot be attributed solely to hard disconnection.

AuthorsOpenReview ↗

Response to W3 & Q3

W3 & Q3. Computational scalability and the role of the pruning budget.

Thank you for these questions. We clarify the computational costs and provide a quantitative reference for the local pruning budget.

(1) Computational cost and scalability.

As detailed in our Complexity Analysis appendix, the time and space complexities of one graph instance are: Time=O ⁣((Ng+Np)dh2+Edh),Space=O ⁣((Ng+Np+E)dh),\mathrm{Time}=\mathcal O\!\left((N _ g+N _ p)d _ h^2+Ed _ h\right),\qquad\mathrm{Space}=\mathcal O\!\left((N _ g+N _ p+E)d _ h\right), where NgN_g and NpN_p are the numbers of gene and pathway nodes, EE is the number of prior edges, and dhd_h is the hidden dimension.

Computation takes place on the shared gene–pathway graph. Increasing the number of cells increases the number of graph instances processed, while the size of each instance remains determined by the gene and pathway sets. With mini-batching, graph memory per batch is governed by this graph size and the batch size; total training time still increases with the number of cells.

Stage I is performed once per context, and Stage II reuses the same sparse propagation operators. The necessity loss adds a counterfactual forward pass during training, whereas ordinary inference requires one forward pass. We supplement this analysis with practical measurements on Adamson. For PerturbDigger, the reported training time includes both calibration stages and perturbation training with the necessity loss.

Computational profile on Adamson using a single NVIDIA RTX PRO 6000 GPU. | Component | Runtime (hours↓) | Peak GPU memory (GB↓) | RMSE↓ |:---|:---:|:---:|:---:| STATE|2.47|43.4|0.079 GEARS|2.04|23.7|0.081 Ours|1.62 (0.78 for calibration)|18.4|0.067

PerturbDigger achieves the lowest RMSE with the shortest total training time and lowest peak GPU memory among the compared methods, including the cost of calibration. This efficiency is supported by sparse, relation-specific message passing over biologically motivated prior edges, with pruning further limiting active mediator interactions and their aggregation costs.

(2) Pruning budget KK.

KK controls local mediator selection: KtgK_{tg} limits the TF inputs retained for each target gene, while KgpK_{gp} limits the gene inputs retained for each pathway. Its definition therefore does not scale with the number of training cells. We provide complementary support from public biological resources and our sensitivity experiments.

External quantitative reference. We analyzed the published CollecTRI regulons [1] and human gene–pathway mappings from Reactome [2]. We counted distinct incoming neighbors after deduplicating relationships, with statistics computed over covered receiving entities.

Incoming-neighbor distributions in public biological resources. CollecTRI uses its frozen version 2.0; Reactome uses release 98. | Resource and receiving entity | Covered entities | Median inputs | Inputs ≤10\leq 10 | Inputs ≤15\leq 15 | |:---|:---:|:---:|:---:|:---:| | CollecTRI: annotated regulators per target gene | 6,693 | 3 | 84.7% | 90.5% | | Reactome: genes per pathway, lowest-level mapping | 2,337 | 11 | 49.9% | 61.7% |

These results show that 90.5% of covered target genes have at most 15 annotated regulators, while 61.7% of lowest-level pathways contain at most 15 annotated genes, with a median of 11. These distributions provide a quantitative rationale for exploring K=5–20K=5–20 as a practical range for local input selection. Since KK is an upper bound, nodes with fewer eligible inputs can retain fewer than KK neighbors. Nevertheless, a fixed local budget may restrict information at high-degree receiving nodes, We will discuss this potential issue in the limitations section.

Empirical sensitivity analysis. As shown in the dedicated sensitivity analysis in Fig. 3, the best-performing budgets fall within the relatively narrow range of K=10–15K=10–15 across the evaluated datasets. This provides empirical support for a compact local propagation budget and is consistent with the biological intuition that a limited set of informative regulators and pathway members can capture the dominant response signals.

[1] Expanding the coverage of regulons from high-confidence prior knowledge for accurate estimation of transcription factor activities.

[2] The reactome knowledgebase 2026.


Thanks again for your time and thoughtful feedback. If you have any additional concerns or suggestions, please feel free to share them, we would be happy to discuss further.

Sincerely,

Reviewer_tMymOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper reframes genetic perturbation prediction as learning from partially trusted priors: pathway/prior graphs are incomplete, noisy, and context-dependent, so they should be calibrated rather than treated as fixed ground truth.

Critical strengths

Treating biological priors as partially trusted and context-specific — calibrating them with control data and enforcing sufficiency on the active mediator subgraph, rather than post-hoc saliency — is a principled shift that matches how pathway knowledge actually behaves, and it yields testable TF/pathway mediators instead of black-box predictions.

The four-part evaluation (counterfactual faithfulness, stability, perturbation-level specificity, gene-level cases) and the K562→H1 few-shot protocol with non-targeting controls test both whether subgraphs drive predictions and whether they transfer.

Critical weaknesses

"Biological validation" is post-hoc and partly circular. Subgraphs are validated against the literature and on canonical pathways that were encoded in the prior graph, and counterfactual masking is done on model hiddens, not real interventions; perturbation-level Jaccard is only modestly above random. Without wet-lab confirmation, "biologically faithful" is stronger than the evidence.

Fairness and significance are under-specified. Prior-based baselines use fixed/uncertain prior versions and may not receive comparable calibration; gains over 10 seeds lack paired significance tests, OOD is a single direction, and the many hyperparameters are only partially analyzed, raising tuning risk on small datasets.

Questions for authors

see weakness

6 comments
AuthorsOpenReview ↗

Response to W1 & Q1

Dear Reviewer tMym,

Thanks for your constructive comments. Below we will address your concerns.


W1 & Q1. Distinguishing model faithfulness from biological validation.

(1) Model faithfulness and biological validation.

Thank you for emphasizing this distinction. We agree that model faithfulness and biological validation are complementary but distinct properties. Jointly ensuring them remains challenging, as alignment with biological ground truth typically requires controlled experiments and extensive wet-lab validation [1]. Nevertheless, PerturbDigger makes this relationship directly testable. Unlike post hoc approaches, PerturbDigger limits propagation to a calibrated, biologically grounded mediator graph whose selected edges directly participate in forward computation. As you noted, “it is a principled shift that matches how pathway knowledge actually behaves, and it yields testable TF/pathway mediators instead of black-box predictions.” This couples biological priors with model decisions, promoting model faithfulness and alignment with biological evidence.

Even so, we will clarify this distinction in the related-work section, temper potentially overstrong claims, and describe the resulting subgraphs more precisely as “model-faithful and biologically supported.” Moreover, we complement our evaluations using independent biological support as follows:

TF–gene binding evidence. Following the ChIP-seq-based GRN evaluation protocol of LINGER [2], we evaluated PerturbDigger’s TF–gene edges against independent binding evidence from ENCODE. For each TF with ChIP-seq coverage, we ranked candidate edges by average gating scores across Adamson test samples, assigning zero to pruned edges. We report the AUPR ratio (AUPR divided by the prevalence of ChIP-supported edges): 11 denotes random ranking, and values above 11 indicate enrichment of supported binding edges.

ENCODE support for TF–gene edge rankings. Variant|median AUPR ratio(↑) :-|:-: Random|1.000 Full model (Ours)|2.758 w/o calibration|1.842

PerturbDigger’s median AUPR ratio of 2.758 exceeds both random ranking and the model without calibration. This provides independent binding evidence for the mediator subgraphs.

Direct mediator-perturbation evidence. We used the distinct Replogle K562 genome-wide CRISPRi screen, which targets all expressed genes and provides their observed transcriptional effects, as an external reference not used in model training or calibration. For each Adamson test perturbation pp, we extracted its selected gene mediator mm and compared the original perturbation effect with the independently measured CRISPRi effect of mm over the mediator’s selected downstream genes. We evaluated affected-gene overlap rather than numerical agreement. We report downstream-overlap enrichment (OE), defined as the observed overlap divided by its random expectation.

Functional support for selected mediators from independent perturbation data. ||mean OE(↑) :-|:-: Random|1.000 Ours|3.659

The greater-than-random enrichment provides independent functional support that the selected mediators affect the expected downstream genes.

[1] Position: Genomic Model Research Must Move Beyond Anecdotal Evaluation of Interpretability Methods. ICML 2026

[2] Inferring gene regulatory networks from single-cell multiome data using atlas-scale external data. NBT 2024


Below, we address your concerns about counterfactual masking and Jaccard similarities.

(2) What the counterfactual masking analysis establishes.

We agree that counterfactual masking is a computational intervention, not a physical intervention in cells. This analysis tests whether the trained predictor depends on the identified biological relationships.

As discussed in (1), the selected edges directly participate in forward computation. This design couples biological priors with model decisions, promoting both model faithfulness and alignment with biological evidence. Thus, establishing computational dependence also reflects the model’s reliance on biological priors and their constraints.

(3) Interpretation of the Jaccard similarities.

We compared learned explanations with random subgraphs from the same candidate graph, matching sizes and relation types and using the same sample grouping and aggregation:

ComparisonObserved JaccardMatched-random Jaccard
CTNNB1: disjoint sample subsets0.740.0012
CTNNB1 versus EP3000.320.0011
CTNNB1 versus PCCB0.130.0008

All three observed similarities exceed the matched-random values by more than two orders of magnitude. They also reveal a biologically meaningful pattern: explanations are highly reproducible across disjoint samples of the same perturbation, more similar between the functionally related perturbations CTNNB1 and EP300, and less similar between CTNNB1 and the functionally distinct perturbation PCCB.

AuthorsOpenReview ↗

Response to W2 & Q2 (Part 1/2)

W2 & Q2. Evaluation fairness, statistical significance, cross-context transfer, and hyperparameter selection.

Thank you for these constructive comments. We agree that the comparison protocols, statistical evidence, and model-selection procedures should be made more explicit. We address these concerns separately below.

(1) Prior resources and calibration fairness.

We thank the reviewer for this concern. All baselines were reproduced and re-evaluated under our experimental protocol. For biological resources shared with PerturbDigger, we used the same resource versions to avoid confounding the comparison with differences in prior versions. Specifically, for GEARS, we used the GO ontology release dated June 15, 2026; for GRAPE, we used Ensembl release 116.

Regarding calibration, these methods do not include the control-based calibration used by PerturbDigger. They also cannot directly use the calibrated graph, as their original formulations do not provide a corresponding mechanism for incorporating the calibrated weights. This calibration therefore represents a methodological contribution and advantage of PerturbDigger, rather than a source of unfairness in the comparison. It is a central component of our framework that directly addresses the potential mismatch between external biological knowledge and the cellular context.

For the general GNN baselines—GCNII, GraphSAGE, and Graph Transformer, we use the same prior graph, calibration procedure, and resulting calibrated edge reliabilities as PerturbDigger. These comparisons therefore evaluate the contribution of the propagation architecture under matched biological resources and calibration.

(2) Paired statistical evaluation across experimental runs.

We have supplemented the reported means and standard deviations with a paired statistical analysis of RMSE across the 10 experimental seeds. For each dataset, we compare PerturbDigger with the strongest competing perturbation-prediction baseline and the strongest GNN baseline using a two-sided exact paired permutation test.

RMSE comparisons across 10 seeds. Values are mean ± standard deviation; lower values indicate better performance. | Model / comparison | Adamson | Norman | Replogle | |:--|:--:|:--:|:--:| | PerturbDigger | 0.067 ± 0.001 | 0.036 ± 0.001 | 0.072 ± 0.001 | | Best prediction baseline | 0.075 ± 0.001 (GRAPE) | 0.048 ± 0.001 (GRAPE) | 0.081 ± 0.001 (STATE) | | Paired permutation pp-value | 0.0020 | 0.0020 | 0.0020 | | Best GNN baseline (GT) | 0.080 ± 0.001 | 0.050 ± 0.001 | 0.081 ± 0.002 | | Paired permutation pp-value | 0.0020 | 0.0020 | 0.0020 |

PerturbDigger achieved lower RMSE than each comparator in all 10 paired runs. Consequently, all six comparisons attained the minimum two-sided exact permutation pp-value of 2/1,024=0.0019531252/1,024=0.001953125. These results provide paired statistical evidence for the consistency of the RMSE improvements across seeds. We will include this analysis and the testing protocol in the revised manuscript.

AuthorsOpenReview ↗

Response to W2 & Q2 (Part 2/2)

(3) Additional cross-context transfer experiments.

We agree that the original K562→H1 experiment represents only one source-to-target transfer. We therefore extend the evaluation using the K562 and RPE1 Perturb-seq screens released by Replogle et al. [1], considering both K562→RPE1 and RPE1→K562.

In each direction, the perturbation predictor is trained exclusively on the source-cell-line data. Before target-context inference, we recalibrate the graph using only target non-targeting controls, while keeping the perturbation-predictor parameters fixed. No target perturbed-cell profiles or perturbation-response labels are used for model training, model selection, or graph calibration. The compared methods follow the same target-label restrictions and have access to the same target non-targeting controls.

Cross-cell-line transfer without target perturbation supervision. | Source → Target | Method | RMSE ↓ | DAC ↑ | PCC ↑ | DES ↑ | PDS ↑ | |:---|:---|:---:|:---:|:---:|:---:|:---:| | K562 → RPE1 | GRAPE | 0.132 | 0.416 | 0.315 | 0.203 | 0.396 | | K562 → RPE1 | PerturbDigger | 0.108 | 0.681 | 0.534 | 0.337 | 0.652 | | RPE1 → K562 | GRAPE | 0.176 | 0.397 | 0.312 | 0.185 | 0.370 | | RPE1 → K562 | PerturbDigger | 0.124 | 0.602 | 0.489 | 0.294 | 0.628 |

PerturbDigger outperforms GRAPE across all five reported metrics in both transfer directions. These experiments broaden the original evaluation and provide additional evidence for cross-context transfer without target perturbation-response supervision.

[1] Mapping information-rich genotype-phenotype landscapes with genome-scale perturb-seq. Cell 2022

(4) Hyperparameter selection and sensitivity.

As described in the manuscript, our hyperparameter grids cover a broad range of reasonable configurations to ensure that each model is adequately evaluated on each dataset. Although a finite grid cannot guarantee globally optimal performance, the search ranges for learning rate, weight decay, hidden dimension, and dropout were chosen to cover settings appropriate for these datasets and consistent with the recommended configurations of the respective baselines.

For the PerturbDigger-specific pruning budget K, we already provide a dedicated sensitivity analysis in the manuscript. KK controls local mediator selection and has a direct biological interpretation: KtgK_{tg} limits the number of TF inputs retained for each target gene, while KgpK_{gp} limits the number of gene inputs retained for each pathway. Its definition therefore does not scale with the number of training cells. Across the evaluated datasets, the best-performing budgets fall within the relatively narrow range of K=10–15K=10–15. This provides empirical support for a compact local propagation budget across the evaluated settings and is consistent with the biological intuition that a limited set of informative regulators and pathway members can capture the dominant response signals.

To further address more concern about PerturbDigger-specific hyperparameters, we conducted a sensitivity analysis over the two regression hyperparameters used in Stage I calibration. Our default configuration uses a Ridge penalty of alpha=10alpha=10 and B=20B=20 base estimators. To assess sensitivity, we vary the Ridge penalty over α∈5,10,20\alpha \in{5,10,20} while fixing B=20B=20, and vary the number of base estimators over B∈10,20,50B\in{10,20,50} while fixing α=10\alpha=10.

Downstream prediction performance and similarity between the Stage I qq vectors, using the Adamson dataset. The first line is the the default configuration. α\alpha|BB|RMSE|Spearman ρ|Jaccard@1000 :-:|:-:|:-:|:-:|:-:| 10|20|0.0673|-|- 10|10|0.0680|0.964|0.894 10|50|0.0671|0.977|0.911 5|20|0.0677|0.975|0.896 20|20|0.0675|0.973|0.905 Avg.|-|-|0.972|0.901

As shown in the table, the prediction RMSE metric varied by at most 0.0007, while the resulting qq vectors retained an average Spearman correlation of 0.972 and a top-1000 edge Jaccard similarity of 0.901 relative to the default configuration. This shows that both the calibration prior and downstream prediction are stable to these hyperparameters.


Thanks again for your time and thoughtful feedback. If you have any additional concerns or suggestions, please feel free to share them, we would be happy to discuss further.

Sincerely,

Reviewer_jZiKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PerturbDigger for predicting gene expression changes following genetic perturbations and generating mechanistic explanations. The method first uses control cells to calibrate a heterogeneous biological prior graph, then makes predictions through ordered propagation among genes, transcription factors (TFs), and pathways. Sparse mediator selection and a necessity loss tie the explanatory edges to the prediction process. Experiments cover unseen perturbation prediction on three K562 datasets and few-shot transfer to H1 hESCs.

Critical strengths

The motivation is clear. Existing biological graphs may not accurately reflect a specific cellular context, so calibrating these priors using control cells is a reasonable approach. Incorporating TFs and pathways as explicit mediators in the prediction process also gives the explanations a clear biological interpretation.

The predictive results are strong. In Table 1, PerturbDigger achieves the best reported mean performance on all three metrics across all three datasets. The comparisons include existing perturbation prediction methods and general-purpose GNNs built using the same biological graph resources. Results are also reported over ten random seeds.

The main text provides substantial ablation studies and explanation analyses. Table 2 examines the contributions of edge types, graph calibration, pruning, and the necessity loss. Figure 4 compares the effects of deleting highly ranked, low-ranked, and randomly selected edges, and evaluates explanation stability. These experiments help assess whether the selected explanations are relevant to the model’s predictions.

Critical weaknesses

It remains unclear how much additional mechanistic information the explanations provide beyond the input priors. The edge-deletion experiments show that the selected edges affect predictions, and the stability analyses show that some explanations are reproducible. However, these results do not distinguish learned mechanistic information from biological knowledge already encoded in the priors. Since regulatory relationships and pathway knowledge are used to construct the graph, I suggest including a simple explanation baseline that selects edges solely from the input priors, with sparsity matched to the learned explanations. This would help establish what the learned explanations add.

The cross-context experiments lack a training-from-scratch baseline under the same target-data budget. Section 4.3 uses a small amount of perturbation data from the target context and recalibrates the graph using target-context non-targeting control cells. The current results therefore make it difficult to determine the additional benefit of source-context pretraining. I suggest comparing pretrained adaptation with training directly on H1 hESCs, and clarifying whether all methods use the same target data.

The Adamson DAC values in Table 4 raise a numerical consistency concern that should be checked as a priority. DAC-DE20 is 0.927, whereas DAC-DE40 is 0.979. Under the paper’s definition of the Direction Accuracy Coefficient (DAC), if both metrics use the same predictions, evaluation samples, and aggregation weights, and the top-20 genes are a subset of the top-40 genes, then even if predictions for all 20 additional genes have the correct direction, the following bound must hold: DAC_40≤(1+DAC_20)/2=0.9635 The current description therefore does not explain the reported value of 0.979, and ordinary rounding cannot account for the discrepancy. The authors should verify the table entries and clarify whether the gene subsets, effective denominators, or aggregation procedures differ.

Questions for authors

Could you include an explanation baseline that selects edges solely from the input priors, with sparsity matched to the learned explanations, to assess what the learned TF and pathway explanations add beyond the priors themselves?

Under the same target-data budget, how does source-context pretraining followed by adaptation compare with training from scratch on H1 hESCs? How much does recalibration using target-context control cells contribute to performance?

Do DAC-DE20 and DAC-DE40 in Table 4 use the same predictions, perturbation sets, and aggregation weights? Are the two gene subsets derived from the same ranking? Please explain why the two Adamson values do not satisfy the bound above, and verify both the table entries and the metric calculations.

5 comments
AuthorsOpenReview ↗

Response to W1 & Q1

Dear Reviewer jZiK,

Thanks for your constructive comments. Below we will address your concerns.


W1 & Q1. Added value of the learned explanations beyond the input priors.

Thank you for this constructive suggestion. We agree that it is important to quantify what the learned explanations add beyond the input priors. We therefore introduce a sparsity-matched prior-only explanation baseline. Moreover, we complement this comparison with external evidence to assess the biological support for the selected relationships and downstream responses.

(1) Comparison with a sparsity-matched prior-only baseline.

PerturbDigger learns which candidate biological relationships to use for a given perturbation and reference context through its propagation and pruning operations. To assess the contribution of this learned selection, following your suggestion, we uniformly sample edges from the original, unpruned prior graph, matching the numbers of retained TF–gene and gene–pathway edges separately for each test sample.

To isolate the contribution of edge selection, we keep the trained predictor, calibrated edge weights, and supporting propagation operations fixed, and replace only the L2/L3 selection masks. Both explanations are evaluated on the same test samples, with random selection repeated to account for sampling variability.

Prediction performance using learned and sparsity-matched prior-only explanation masks on Adamson. | Explanation selection | RMSE ↓ | DAC ↑ | PCC ↑ | |:---|:---:|:---:|:---:| | Prior-only, matched sparsity | 0.216 | 0.517 | 0.489 | | Learned selection | 0.067 | 0.786 | 0.801 |

The sparsity-matched prior-only baseline consistently degrade the performance across RMSE, DAC, and PCC. These comparisons directly assess whether the learned selector identifies more prediction-relevant propagation routes than retaining the same number of edges from the input priors.

(2) External support for the selected relationships and downstream responses.

We further assess the biological support for the explanations using two complementary sources of external evidence.

(2.1) TF–gene binding evidence. Following the ChIP-seq-based GRN evaluation protocol of LINGER [1], we evaluated PerturbDigger’s TF–gene edges against independent binding evidence from ENCODE. For each TF with ChIP-seq coverage, candidate edges were ranked by their mean gating scores across Adamson test samples, with pruned edges assigned zero. We report the AUPR ratio (AUPR normalized by the prevalence of ChIP-supported edges), where 11 represents random ranking and values above 11 indicate enrichment of experimentally supported binding edges.

ENCODE support for TF–gene edge rankings on Adamson. Variant|median AUPR ratio(↑) :-|:-: Random|1.000 Full model (Ours)|2.758 w/o calibration|1.842

PerturbDigger achieved a median AUPR ratio of 2.758, substantially exceeding both random ranking and the model without calibration. This provides independent binding evidence for the mediator subgraphs.

(2.2) Direct mediator-perturbation evidence. We used the distinct Replogle K562 genome-wide CRISPRi screen, which targets all expressed genes and provides their observed transcriptional effects, as an external reference not used in model training or calibration. For each Adamson test perturbation pp, we extracted its selected gene mediator mm and compared the original perturbation effect with the independently measured CRISPRi effect of mm over the mediator’s selected downstream genes. We evaluated affected-gene overlap rather than numerical agreement. We report downstream-overlap enrichment (OE), defined as the observed overlap divided by its random expectation.

Functional support for selected mediators from independent perturbation data. ||mean OE(↑) :-|:-: Random|1.000 Ours|3.659

The greater-than-random enrichment provides independent functional support that the selected mediators affect the expected downstream genes.


[1] Inferring gene regulatory networks from single-cell multiome data using atlas-scale external data. Nature Biotechnology 2024

AuthorsOpenReview ↗

Response to W2 & Q2

W2 & Q2. Separating the contributions of source pretraining and target-context recalibration.

Thank you for this constructive suggestion. We agree that a matched training-from-scratch comparison is needed to isolate the benefit of source-context pretraining, and that the contribution of target-context graph recalibration should also be evaluated separately. We therefore extend the H1 hESC evaluation to target training budgets of (b\in{50,100,150}) perturbation conditions. At each budget, all compared variants use identical target training perturbations and cell samples, the same available non-targeting controls, and the same held-out validation and test sets. The training subsets are nested across budgets, while the validation and test sets remain fixed.

We compare four variants while retaining the same model architecture, biological priors, and perturbation-training objective:

  • Training-from-scratch, with target calibration. We initialize the perturbation predictor randomly and train it directly on the selected H1 hESC training data. This variant uses no source-trained predictor parameters, but retains graph calibration using H1 non-targeting controls.
  • Training-from-scratch, without calibration. We use the same source but retain the edge reliabilities as unit weight.
  • Pretrained adaptation, with target recalibration. We initialize the predictor from the source-pretrained checkpoint (Adamson K562 and Replogle K562) and fine-tune it on exactly the same H1 training data. This variant uses the same target-control-calibrated graph as the scratch variant, thereby isolating the contribution of source pretraining.
  • Pretrained adaptation, without target recalibration. We use the same source-pretrained initialization but retain the source-calibrated edge reliabilities throughout target adaptation.

Both scratch training and pretrained adaptation receive sufficient training to converge, with comparable hyperparameter-search budgets, and the same validation-based checkpoint-selection criterion.

H1 hESC performance under matched target training budgets. Each entry reports RMSE↓. training budgets |Training from scratch, without target calibration| Training from scratch, with target calibration |Pretrained, without target recalibration|Pretrained, with target recalibration| :-:|:-:|:-:|:-:|:-: 50 |0.191 |0.179 | 0.115 | 0.082 100 |0.118 |0.106 | 0.081 | 0.060 150 |0.091 |0.085 | 0.069 | 0.058

The results reveal two complementary findings:

  • Contribution of source-context pretraining. With target calibration held fixed, pretraining reduces RMSE by 54.2%, 43.4%, and 31.8% at budgets of 50, 100, and 150 perturbations, respectively. With only 50 target perturbations, pretrained adaptation achieves a mean RMSE of 0.082, compared with 0.085 for training from scratch using 150 perturbations, supporting improved target-data efficiency.
  • Contribution of target-context recalibration. For pretrained models, target recalibration reduces RMSE by 28.7%, 25.9%, and 15.9% across the three budgets. Calibration also improves scratch-training performance at every budget, demonstrating its benefit even without source-context predictor pretraining.

We will incorporate these results into Section 4.3 and explicitly document the target training, validation, and control data available to each baseline in the implementation details.

AuthorsOpenReview ↗

Response to W3 & Q3

W3 & Q3. Consistency of the DE20 and DE40 evaluation.

Thank you for identifying this issue. We apologize for any confusion caused by the inconsistent reporting.

The DE20 and DE40 metrics were added in different rounds of supplementary experiments after completion of the main manuscript. Therefore, for each dataset, the original DAC/PCC-DE20 and DAC/PCC-DE40 results were obtained from two independent sets of experimental runs, rather than jointly from the same predictions. This distinction was not stated in the manuscript, and the table could be read as reporting paired evaluations.

We have now repeated the supplementary evaluation and computed DE20 and DE40 jointly within each run. For each perturbation, we construct a single observed differential-expression ranking and select its top 20 and top 40 genes, ensuring that the former is a subset of the latter. Both evaluations use identical predictions, perturbation sets, and aggregation weights. We report the updated results in a dedicated table, explicitly identifying them as a new set of supplementary runs conducted separately from the original main experiments.

Corrected DEG-focused evaluation results computed jointly under the unified evaluation protocol, averaged over 10 random seeds. | Model | DAC-DE20 ↑ | DAC-DE40 ↑ | PCC-DE20 ↑ | PCC-DE40 ↑ | |:---|:---:|:---:|:---:|:---:| | Adamson | 0.925±0.003 | 0.937±0.002 | 0.849±0.004 | 0.882±0.003 | | Norman | 0.865±0.004 | 0.884±0.003 | 0.677±0.003 | 0.696±0.002 | | Replogle | 0.768±0.007 | 0.831±0.008 | 0.819±0.008 | 0.834±0.009 | | VCC | 0.836±0.001 | 0.862±0.002 | 0.773±0.002 | 0.814±0.002 |

Thank you for helping us identify and correct this reporting problem. We have also reviewed the experimental settings and reporting of the remaining results and found no further errors or inconsistencies.


Thanks again for your time and thoughtful feedback. If you have any additional concerns or suggestions, please feel free to share them, we would be happy to discuss further.

Sincerely,

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revisions Summary

Dear Reviewers and Area Chair,

Thank you again for your constructive feedback. We have revised the manuscript to address the key concerns raised during the discussion. The main revisions are summarized below.


1. Clarified biological validity and added external experimental evidence.

We clarified the distinction between model faithfulness and biological validity in Section 4.5 (Lines 489–497) and added Appendix C with independent biological evidence and further discussion (tMym: W1/Q1; jZiK: W1/Q1; KgBa: Q2).

2. Corrected and unified the DEG-focused evaluation.

We clarified the evaluation protocol and reran the DE20 and DE40 experiments jointly across 10 random seeds. The corrected results are reported separately in Appendix B.4 (jZiK: W3/Q3).

3. Added controlled ablations and cross-context generalization experiments.

We expanded the appendix with additional controlled ablations, robustness analyses, and cross-context OOD experiments (Appendix B.5, Appendix D) (jZiK: W2/Q2; tMym: W2/Q2; KgBa: W1/Q1).


The other questions are responsed in our point-by-point replies. We hope that our replies and revisions could help resolve your concerns. If there are any further questions or suggestions, we would be happy to discuss them with you. Thank you again for your time and thoughtful feedback.

Sincerely,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.