- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper presents CellReprogrammer, a framework for selecting small gene intervention sets using a disease classifier, a perturbation surrogate, and differentiable top-K optimization. The objective combines predicted disease-state reversal with drug-realizability and target-risk penalties, while capping downstream expression changes. Experiments across five disorders and three cell types report improvements over 20 baselines. However, the evidence remains computational, and the current evaluation does not sufficiently establish therapeutic relevance or reliable prediction of combinatorial effects.
Critical strengths
- The problem is relevant: selecting compact intervention sets while considering effectiveness, feasibility, and risk is a useful research direction.
- The framework integrates these objectives coherently and includes several sensible controls, such as donor-level aggregation and perturbation-response differencing.
- The experiments cover multiple disease–cell-type settings and include ablations of the objective, surrogate, and optimization constraints.
Critical weaknesses
- The evaluation lacks independence. Interventions are optimized and assessed through the same predictive pipeline, leaving open whether the optimizer exploits model errors rather than identifies meaningful disease reversal. No experimental intervention validation is provided.
- Surrogate validation does not support the full search space or combination claims. Measured perturbations cover only a small subset of the 3,000 candidates per cell type, and held-out single-target prediction does not establish accuracy for multi-target interventions. The description of weighted response vectors also leaves the modeling of non-additive interactions unclear.
- The baseline comparisons are not sufficiently controlled. Training protocols and donor holdout procedures differ across methods, and some baselines use zero-shot predictions or surrogate-generated training data. Matched computational budgets and stronger statistical comparisons are needed.
- The presentation significantly limits accessibility. Figure 1 can not be smoothly read.
Questions for authors
- Can the authors evaluate selected combinations using an independent predictor?
- How accurately does the surrogate predict unseen multi-target interventions, including response magnitudes and non-additive effects?
- Do the reported gains persist under consistent donor-held-out protocols, matched training and computational budgets, and an analysis that includes unsuccessful effect-matching cases?
Independent Evaluation, Multigene Prediction, and Effect Matching
Q1: Can the authors evaluate selected combinations using an independent predictor?
A1: Thank you for this suggestion. We added two complementary evaluations: (i) rescoring selected target sets in the original cohort with four alternative evaluators—RF, MLP, scPRINT2, and CellWhisperer—and (ii) evaluating selected target sets in independent cohorts with five evaluators—LR and the same four alternatives. These comparisons separately examine sensitivity to the disease-scoring model and transfer to new cohorts.
Alternative evaluators. At K = 3, we retained the selected target sets and rescored the predicted perturbed profiles using RF, MLP, scPRINT2, and CellWhisperer without reselecting targets. Table R1 reports mean control-adjusted logit changes; more negative values indicate stronger predicted effects. CellReprogrammer outperformed the strongest baseline under each evaluator (55–78% improvement, Table R1).
Independent cohorts. We evaluated selected targets on external disease-donor profiles (ASD: GSE298243; BPD: GSE306819; MDD: GSE213982; PTSD: GSE313257; SCZ: GSE254569), using the same control-adjusted scoring procedure with LR and the four alternative evaluators. CellReprogrammer outperformed the strongest baseline under each evaluator (57–72% improvement, Table R1).
Table R1. Predicted reprogramming effects across evaluators and cohorts
Q2: How accurately does the surrogate predict unseen multi-target interventions, including response magnitudes and non-additive effects?
A2: Thank you for raising this question. We evaluated the surrogate on 26 held-out two-gene perturbations in K562 cells. Joint prediction reduced MSE by 52.4% relative to the additive baseline across all 3,000 genes (0.003102 vs. 0.006510) and increased mean Pearson r from 0.7643 to 0.8458 (Table R2). The joint model also achieved lower MSE and higher Pearson r on the top 20, 50, and 100 response genes. Genes were ranked by absolute measured expression change, with metrics averaged equally across the 26 combinations.
For our three benchmark cell types (excitatory neurons, microglia, and astrocytes), suitable Perturb-seq datasets with matched single-gene and multigene measurements remain unavailable, limiting direct validation in these settings.
Table R2. Prediction accuracy by response gene set
Q3: Do the reported gains persist under consistent donor-held-out protocols, matched training and computational budgets, and an analysis that includes unsuccessful effect-matching cases?
A3: Thank you for raising these points. Across all 510 folds, phenotype guidance reached the predicted-effect threshold more often than expression matching (492/510 vs. 322/510; 96.5% vs. 63.1%, a 33.3-percentage-point improvement).
Consistent donor holdout. CellReprogrammer and direct-search baselines use the same training donors for target selection and the same held-out donor for evaluation. PDGrapher uses surrogate-predicted post-treatment profiles for training because paired treatment measurements were unavailable.
Effect-matching coverage. At Δℓadj=−0.1, all 510 folds are included in Table R3. Figure 4A compares non-target expression changes within the 314 folds where both methods reached the same predicted effect.
Table R3. Effect-matching outcomes across all 510 folds
Across four matched-effect levels (Δℓadj=−0.05,−0.1,−0.2,−0.3), phenotype guidance showed fewer large non-target expression changes within jointly successful folds. The all-fold analysis separately reports how often each effect was attainable.
Computational budgets. On one MDD–EXN donor-held-out fold (3,000 candidates, RTX 3090), CellReprogrammer took 1.04–2.09 s for K = 1–20, versus 0.41–8.38 s for greedy. Random and genetic search took less than 0.4 s, and simulated annealing ~1.1 s. PDGrapher required ~26 min for proxy-data preparation and retraining, reused across K.