- Rating
- 2
- Confidence
- 1
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes PerturbDigger, a graph learning framework designed for genetic perturbation prediction and mechanistic explanation. To address the limitations of treating biological priors as fixed ground truth and lacking explicit mechanistic reasoning, PerturbDigger formulates biological prior knowledge as a partially trusted prior graph, performs context-specific structural calibration using control data, and predicts perturbation responses through a biologically-guided graph transformer regularized by faithfulness constraints on mechanistic subgraphs. Experiments across multiple benchmark datasets and a real-world cell context generalization benchmark demonstrate improved predictive performance and stable, faithful, and biologically meaningful mechanistic explanations.
Critical strengths
- The formulation of biological priors as partially trusted graphs combined with context-specific structural calibration effectively bridges the gap between rigid prior structures and context-dependent regulatory relationships.
- The biologically-guided graph transformer enforces ordered propagation through explicit biological mediators (such as TFs and pathways), offering intrinsic interpretability rather than relying solely on post hoc attributions.
- The incorporation of faithfulness constraints (via counterfactual necessity) ensures that the learned mechanistic subgraphs are genuinely responsible for model predictions.
Critical weaknesses
- The methodological framework heavily relies on the quality and coverage of external biological knowledge bases (such as CellOracle and Reactome). For poorly annotated genes or non-model organisms, the reliability of the partially trusted prior graph and calibration steps may degrade significantly.
- While the paper emphasizes faithfulness via counterfactual necessity, the evaluation of the biological mechanisms heavily relies on qualitative case studies (e.g., literature-consistent subgraphs for CTNNB1 and EP300), lacking systematic quantitative validation against independent large-scale wet-lab perturbation benchmarks.
- The computational overhead introduced by the multi-stage structural calibration and relation-specific constraints is insufficiently discussed regarding its scalability to genome-scale networks with hundreds of thousands of cells and diverse cell lines.
Questions for authors
- How sensitive is PerturbDigger's performance to the choice and noise level of initial biological priors (e.g., missing edges in CellOracle or Reactome)? Can the calibration stage successfully recover heavily corrupted or entirely missing regulatory pathways?
- In Section 3.3, the necessity loss relies on counterfactual masking of mediator edges. Since masking edges in a graph transformer can lead to distribution shifts, how does the model disentangle true mechanistic necessity from artifacts caused by breaking message-passing connectivity?
- Could the authors clarify how the pruning budget parameter is tuned across datasets of varying sizes, and whether a fixed leads to structural bias when explaining perturbations that involve vastly different numbers of downstream target genes?
7 comments
Response to W2 & Q2
W2. Systematic quantitative validation using independent experimental data.
Thank you for this suggestion. We complement the qualitative examples with systematic evaluations using independent binding and perturbation data.
TF–gene binding evidence. Following the ChIP-seq-based GRN evaluation protocol of LINGER [2], we evaluated PerturbDigger’s TF–gene edges against independent binding evidence from ENCODE. For each TF with ChIP-seq coverage, candidate edges were ranked by their mean gating scores across Adamson test samples, with pruned edges assigned zero. We report the AUPR ratio (AUPR normalized by the prevalence of ChIP-supported edges), where represents random ranking and values above indicate enrichment of experimentally supported binding edges.
ENCODE support for TF–gene edge rankings on Adamson. Variant|median AUPR ratio(↑) :-|:-: Random|1.000 Full model (Ours)|2.758 w/o calibration|1.842
PerturbDigger achieved a median AUPR ratio of 2.758, substantially exceeding both random ranking and the model without calibration. This provides independent binding evidence for the mediator subgraphs.
Direct mediator-perturbation evidence. We used the distinct Replogle K562 genome-wide CRISPRi screen, which targets all expressed genes and provides their observed transcriptional effects, as an external reference not used in model training or calibration. For each Adamson test perturbation , we extracted its selected gene mediator and compared the original perturbation effect with the independently measured CRISPRi effect of over the mediator’s selected downstream genes. We evaluated affected-gene overlap rather than numerical agreement. We report downstream-overlap enrichment (OE), defined as the observed overlap divided by its random expectation.
Functional support for selected mediators from independent perturbation data. ||mean OE(↑) :-|:-: Random|1.000 Ours|3.659
The greater-than-random enrichment provides independent functional support that the selected mediators affect the expected downstream genes.
Q2. Distinguishing predictive dependence from artifacts of graph disruption.
Thank you for raising this concern. PerturbDigger constrains propagation to a calibrated, biologically grounded mediator graph whose selected edges directly participate in forward computation. Within our model, dependence on these mediators is therefore implemented through their message-passing contributions. This design couples biological priors with model decisions, promoting both model faithfulness and alignment with biological evidence.
To address your concerns, we introduce two additional controls to examine whether the identified relationships contribute beyond the effects of disrupting connectivity. Both experiments use Adamson with a fixed intervention budget of 30% of the originally active mediator edges:
- Structurally matched deletion. We compare top-ranked edge deletion with random deletion from the originally active mediator edges, matching the number of edges removed for each relation type at each receiving node. This controls both the amount and location of input loss, allowing us to assess whether top-ranked relationships contribute more than structurally matched alternatives.
- Attenuation without removing connections. Using the same edge sets, we halve their gated message contributions instead of deleting them. This preserves the selected connections while reducing their influence, testing whether the observed dependence persists without complete disconnection.
Additional controls on Adamson. Entries report RMSE increases () relative to the original prediction. Both interventions affect 30% of the active mediator edges.** | Intervention | Top-ranked edges | Matched-random edges | |:---|:---:|:---:| | Edge deletion | 0.021 | 0.006 | | Message attenuation by 50% | 0.015 | 0.003 |
Top-ranked interventions produce larger RMSE increases than the structurally matched random controls under both deletion and message attenuation. The persistence of this difference when connections are retained supports predictive dependence on the selected relationships and indicates that the observed effects cannot be attributed solely to hard disconnection.
Response to W3 & Q3
W3 & Q3. Computational scalability and the role of the pruning budget.
Thank you for these questions. We clarify the computational costs and provide a quantitative reference for the local pruning budget.
(1) Computational cost and scalability.
As detailed in our Complexity Analysis appendix, the time and space complexities of one graph instance are: where and are the numbers of gene and pathway nodes, is the number of prior edges, and is the hidden dimension.
Computation takes place on the shared gene–pathway graph. Increasing the number of cells increases the number of graph instances processed, while the size of each instance remains determined by the gene and pathway sets. With mini-batching, graph memory per batch is governed by this graph size and the batch size; total training time still increases with the number of cells.
Stage I is performed once per context, and Stage II reuses the same sparse propagation operators. The necessity loss adds a counterfactual forward pass during training, whereas ordinary inference requires one forward pass. We supplement this analysis with practical measurements on Adamson. For PerturbDigger, the reported training time includes both calibration stages and perturbation training with the necessity loss.
Computational profile on Adamson using a single NVIDIA RTX PRO 6000 GPU. | Component | Runtime (hours↓) | Peak GPU memory (GB↓) | RMSE↓ |:---|:---:|:---:|:---:| STATE|2.47|43.4|0.079 GEARS|2.04|23.7|0.081 Ours|1.62 (0.78 for calibration)|18.4|0.067
PerturbDigger achieves the lowest RMSE with the shortest total training time and lowest peak GPU memory among the compared methods, including the cost of calibration. This efficiency is supported by sparse, relation-specific message passing over biologically motivated prior edges, with pruning further limiting active mediator interactions and their aggregation costs.
(2) Pruning budget .
controls local mediator selection: limits the TF inputs retained for each target gene, while limits the gene inputs retained for each pathway. Its definition therefore does not scale with the number of training cells. We provide complementary support from public biological resources and our sensitivity experiments.
External quantitative reference. We analyzed the published CollecTRI regulons [1] and human gene–pathway mappings from Reactome [2]. We counted distinct incoming neighbors after deduplicating relationships, with statistics computed over covered receiving entities.
Incoming-neighbor distributions in public biological resources. CollecTRI uses its frozen version 2.0; Reactome uses release 98. | Resource and receiving entity | Covered entities | Median inputs | Inputs | Inputs | |:---|:---:|:---:|:---:|:---:| | CollecTRI: annotated regulators per target gene | 6,693 | 3 | 84.7% | 90.5% | | Reactome: genes per pathway, lowest-level mapping | 2,337 | 11 | 49.9% | 61.7% |
These results show that 90.5% of covered target genes have at most 15 annotated regulators, while 61.7% of lowest-level pathways contain at most 15 annotated genes, with a median of 11. These distributions provide a quantitative rationale for exploring as a practical range for local input selection. Since is an upper bound, nodes with fewer eligible inputs can retain fewer than neighbors. Nevertheless, a fixed local budget may restrict information at high-degree receiving nodes, We will discuss this potential issue in the limitations section.
Empirical sensitivity analysis. As shown in the dedicated sensitivity analysis in Fig. 3, the best-performing budgets fall within the relatively narrow range of across the evaluated datasets. This provides empirical support for a compact local propagation budget and is consistent with the biological intuition that a limited set of informative regulators and pathway members can capture the dominant response signals.
[1] Expanding the coverage of regulons from high-confidence prior knowledge for accurate estimation of transcription factor activities.
[2] The reactome knowledgebase 2026.
Thanks again for your time and thoughtful feedback. If you have any additional concerns or suggestions, please feel free to share them, we would be happy to discuss further.
Sincerely,
Response to W1 & Q1
Dear Reviewer KgBa,
Thanks for your constructive comments. Below we will address your concerns.
W1 & Q1. Dependence on biological priors and robustness to incomplete or corrupted knowledge.
Thank you for highlighting this limitation. We agree that PerturbDigger depends on the coverage and quality of its candidate biological graph. However, our focus is to explain how perturbation effects propagate through biologically testable mediators, specifically, how biological priors can be calibrated to a context-specific cellular environment and used to derive an explanation of how the perturbation occurs, rather than to discover entirely new pathways from perturbation data. Consequently, if an edge is entirely absent from the candidate set, that exact edge cannot appear in the resulting mechanistic subgraph. We will explicitly clarify this scope and limitation in the revised Limitations section.
To evaluate the robustness to incomplete or corrupted priors, We further evaluate these two cases below.
(1) Robustness to incomplete priors.
We randomly removed 10%, 20%, and 30% of the TF–gene edges before calibration and retrained the complete pipeline. We evaluated predictive performance and external ENCODE support using the same evaluation protocol (details shown in Q2).
Performance degrades progressively as more prior edges are removed. Nevertheless, the remaining relationships retain greater-than-chance ENCODE support even after 30% removal. These results demonstrate tolerance to moderate prior incompleteness while confirming that adequate candidate coverage remains important.
(2) Recovery from corrupted edge-reliability assignments.
We further examined whether recalibration can restore useful reliability assignments when the candidate relationships remain available but their weights are corrupted. Starting from the calibrated graph, we constructed two variants while keeping the candidate topology and trained perturbation predictor fixed:
For A-re and B-re, we reran Stages I and II using the same control data, with Stage II initialized from the corresponding corrupted weights. The clean edge weights were not restored, and the perturbation predictor remained fixed.
Both corruption schemes substantially degrade prediction performance, whereas recalibration restores most of the lost performance. These results support the ability of control-based calibration to recover useful reliability assignments within the existing candidate topology.
(3) Contribution of our recovered edges to interpretation.
We additionally evaluated the explanation-verbalization stage on 50 held-out cases, using three experts and an LLM judge. For each case, we compared LLM-generated interpretations of the extracted mechanistic subgraph with interpretations generated after removing all edge relations while retaining the same nodes. The evaluators assessed the paired interpretations in a blinded comparison.
Interpretations based on the original subgraphs were preferred in 84%–92% of cases across evaluators. The experts observed more unsupported or biologically implausible regulatory and causal statements in the node-only condition. This sanity check indicates that the selected relational structure carries essential mechanistic information and helps constrain the resulting biological interpretation.