acceptodds
Under review as a conference paper at ICLR 2027

VCDesign: Finite-Budget Intervention Design for Virtual Cells

Abstract

Virtual-cell models are typically judged by how well they predict the effects of perturbations or recover the interventions associated with a cellular transition. Yet experimental planning poses a different question: given a desired cellular state, which interventions should be tested when only a limited number of experiments can be performed? Here we introduce VCDesign, which, to our knowledge, is the first framework to formulate target-conditioned cellular intervention design as finite-budget ranking over a variable candidate set. Given source and target states, a candidate library and an experimental budget, VCDesign prioritizes the interventions to test. We also introduce a decision-aligned evaluation framework that measures the quality of the selected candidates using independently measured held-out outcomes. For candidates lacking historical perturbation responses, we develop VCDesign-CED, which uses existing perturbation measurements and biological knowledge to infer candidate effects and combine them with direct candidate scoring. Across four CRISPRi Perturb-seq contexts, VCDesign-CED improves the prioritization of response-unseen candidates, increasing pooled budgeted utility at 20 selections by 0.049 and recovering 1.11 additional top-5% candidates on average. When candidate responses are already available in other contexts, direct retrieval performs better, and routing candidates according to response availability further improves mixed-library selection. Controlled analyses further show that transferable perturbation evidence and candidate knowledge contribute more to design performance than simply adding training contexts or increasing effect-model complexity, while prediction accuracy does not reliably rank nearby models by their utility for intervention design. Together, these results establish a task and evaluation framework for using virtual-cell models to make experimental decisions under finite budgets.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
cellular intervention design, finite-budget ranking, virtual cells, perturbation modeling, single-cell genomics, decision-aligned evaluation, AI for biology

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8aDpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses cellular intervention selection under limited experimental budgets by introducing VCDesign, a framework that formulates the task as budget-constrained ranking over candidate pools toward desired cellular states. For candidates lacking historical measurements, the authors propose VCDesign-CED, which predicts intervention effects by combining perturbation data with biological priors and integrates these predictions with direct candidate scoring. When candidate-specific responses are masked, VCDesign-CED achieves a median improvement of +0.0492 in aggregated paired BU@20 over base scorers. Conversely, direct retrieval performs better when historical responses exist, and routing solvers by data availability improves selection quality on mixed pools. Controlled experiments further characterize the impact of measurement scale and biological knowledge on retrospective directional ranking.

Critical strengths

Strengths

  1. The evaluation framework guides method selection across different information regimes. Under a unified intervention target and experimental budget, the paper compares ranking methods when candidate-specific responses are available versus unavailable, examining atlas scale and candidate coverage through controlled experiments. Direct retrieval outperforms CED when historical responses are accessible, and selecting methods based on response availability improves mixed-pool performance. These results clarify how existing measurements influence ranking gains and help practitioners select appropriate methods based on available data.
  2. Controlled experiments confirm practical gains from effect prediction in candidate ranking. In the pre-locked K562 benchmark, CED achieves a median paired BU improvement of +0.0319 over base scorers, with a 95% confidence interval of [+0.0282, +0.0378]. Cross-context gains persist even after excluding candidates used during base scorer training and development. These results support supplementing direct candidate scoring with perturbation history and biological priors, offering empirical justification for intervention screening when candidate-specific responses are unavailable.
Critical weaknesses

Weaknesses

  1. The public implementation of ridge regression cross-validation fails to enforce the identity-level isolation claimed in the paper. In the multi-context training pool, a single gene can span multiple row entries. However, _cross_validate invokes identity_folds over row counts, which partitions row indices uniformly at random without grouping by gene identity. Consequently, records for the same intervention across different contexts leak simultaneously into both training and validation folds. Furthermore, the response SVD basis is fitted on the entire training pool prior to inner-loop splitting, thereby absorbing the inner validation targets. This departs from the claimed unseen-identity validation setting for tuning regularization strength. Grouping splits by gene identity and refitting the response basis strictly within each training fold are necessary to verify parameter choices and the robustness of reported gains.

  2. Candidate pool normalization alters relative rankings of existing candidates, and the public pool-size experiment diverges from the methodological definition. Equation (5) standardizes both branches with respect to the current candidate pool, making their relative weighting dependent on pool composition. A reproducible numerical counterexample confirms that appending candidates inferior on both branches can invert the ranking between the original top two choices. In contrast, the public script poolsize.py computes fused scores over the entire candidate pool before subsetting for evaluation, omitting per-subset re-normalization. Because these two computation orders can yield different top selections, the scale experiment does not directly validate the pool-dependent scoring behavior defined in the paper, leaving the stability of candidate selection under the actual formulation unverified.

Questions for authors

Questions

  1. Implementation verification and critical checks. Does the multi-context ridge regression code correspond to the reported results? How do the regularization strength selection and primary BU gains change after grouping the inner cross-validation by gene identity and fitting the response basis strictly on each training fold?

  2. Do the performance gains of CED hold if the response normalization scheme is selected solely on the training or validation set and then evaluated on an independent test set?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 8aDp (1/2)

Thank you for checking our implementation this closely. Both weaknesses you found were real, and both are fixed. In summary:

  • W1 and Q1, cross-validation: folds are now grouped by gene and the basis is refitted inside each fold. The selected penalty does not change, so every reported BU@20 value is unchanged (gain +0.0492 [+0.0456, +0.0534]).
  • W2, normalization: Eq. 5 now describes the normalization as implemented. The normalization the submitted text described gives a slightly higher gain (+0.0548), the pool-size conclusion holds when each subset is re-normalized, and the top-20 selections barely change.
  • Q2, selection: a selection that never reads the held-out data makes the published choice, so the gain stays +0.0492.

The revised PDF and the code archive in the supplementary material contain all changes.

Numbers are median paired BU@20 gains of VCDesign-CED over the base scorer (higher is better), on candidates whose own responses are withheld from all training contexts, pooled over the three held-out cell lines. Brackets are 95% cluster-bootstrap intervals over query genes.

W1. "The public implementation of ridge regression cross-validation fails to enforce the identity-level isolation claimed in the paper."

Correct on both points. We fixed both, and no reported result changes.

  • What was wrong: folds were cut over (context, gene) rows, so a gene measured in several training contexts could sit on both sides of a fold, and the response SVD basis was fitted before the folds were cut.
  • Fix: the two changes you name, gene-grouped folds and a basis refitted from each fold's training rows only.
  • Effect: the selected penalty is λ=104\lambda = 10^4 in all 36 fits before and after the fix, an interior point of the 10−210^{-2} to 10510^5 grid. The fitted models and every BU@20 value are therefore identical. The leak only inflated the cross-validation cosine (RPE1: 0.1591 before, 0.1500 after, at 10410^4).
  • Reach of the leak: penalty selection inside the training contexts only. BU@20 is always graded in held-out cell lines, and the headline gain uses candidates withheld from all training contexts.

Changes. Section 4 and Appendix G.5. In the code, four_context_v1.predictors.fit now uses selection="grouped" by default (revision_v1/grouped_fit.py, which raises an error if a gene falls on both sides of a fold). selection="frozen" keeps the submitted path so the submitted numbers reproduce exactly, and revision_v1/test_grouped_fit.py tests that grouped folds never split a gene and that both paths give the same predictor at one penalty.

W2. "Candidate pool normalization alters relative rankings of existing candidates, and the public pool-size experiment diverges from the methodological definition."

Correct. The gap was wider than poolsize.py, and fixing it changes no conclusion.

  • What was wrong: every analysis z-scored each branch over the full context pool (RPE1 1,483, HepG2 881, Jurkat 1,696 candidates) and graded the eligible subset (1,273, 764, 1,450), while the Eq. 5 text referred to the current pool.
  • Main result: with z-scores over the eligible pool, as the submitted text defined it, the gain is +0.0548 [+0.0508, +0.0580], against +0.0492 as implemented. We keep the implemented version as the method of record: every submitted number used it, it gives the lower gain, and switching to the higher one after seeing both would be a choice made on held-out results.
  • Pool-size experiment (K562 stress test, response-unseen candidates, gain in BU‾\overline{\mathrm{BU}}, the mean of BU@10, 20 and 50): with re-normalization inside each subset, the curve keeps its shape. The gain still falls beyond 1,000 candidates and is not positive at 6,255.
Candidates2505001,0001,5703,0006,255
Re-normalized per subset+0.0366+0.0466+0.0537+0.0290+0.0147-0.0026
As originally computed+0.0329+0.0416+0.0472+0.0198+0.0057-0.0103
  • Your counterexample: we agree that per-pool z-scoring makes the branch weights depend on pool composition, and Section 4 now says so. Our two normalizations are that operation at real scale, with 117 to 246 extra candidates in the standardization pool. The selections barely change: the two top-20 lists share 0.963 of their candidates on average (median 19 of 20), are identical in 48% of query-view rows, and differ in the top candidate in 5.2%. Adding the effect branch itself changes the top 20 far more (mean overlap 0.666).

Changes. The text under Eq. 5 (Section 4) and Appendix G.5. In the code, four_context_v1.evaluate.fuse(..., standardize_over=mask) standardizes over the given candidates only, and poolsize.py --standardize subset re-normalizes inside each subset. The defaults reproduce the published numbers.

AuthorsOpenReview ↗

Response to Reviewer 8aDp (2/2)

Q1. "Does the multi-context ridge regression code correspond to the reported results? How do the regularization strength selection and primary BU gains change after grouping the inner cross-validation by gene identity and fitting the response basis strictly on each training fold?"

Yes, and neither changes. Rebuilding the held-out results from the code reproduces the published per-query rows with maximum error 0.0. After the fix, the penalty stays 10410^4 in all 36 fits and every BU@20 value is unchanged (+0.0492 [+0.0456, +0.0534]). Details of the fix are under W1 and in Appendix G.5.

Q2. "Do the performance gains of CED hold if the response normalization scheme is selected solely on the training or validation set and then evaluated on an independent test set?"

Yes. A selection that never reads the held-out data makes the published choice, so the gain stays +0.0492 [+0.0456, +0.0534]. With the other choice it would be +0.0454 [+0.0419, +0.0497], so the gain holds either way.

  • What was selected: we read "response normalization scheme" as CED's training target, unit-normalized or raw responses, which the submission chose on the held-out pools.
  • How: a preregistered nested selection. For each held-out context, the choice was made on the other two non-K562 contexts, each run under the same masked protocol. The held-out context's data were never loaded.
  • Result: pooled over its two inner contexts, unit targets lead for every held-out context (RPE1 +0.0065 [+0.0048, +0.0079], HepG2 +0.0039 [+0.0023, +0.0054], Jurkat +0.0007 [-0.0010, +0.0026], no detectable difference; Table 36). Every selection therefore picks the published choice, and the per-context gains are the published ones (RPE1 +0.0526, HepG2 +0.0698, Jurkat +0.0390).
  • If you meant the fusion normalization: it has no fitted parameter, and the alternative gives +0.0548 (W2).
  • Scope: the margins are small, so this shows that the published choice is what a selection-clean procedure picks, not that unit targets are better in general. All K562 choices were made on the training and development splits.

Changes. Section 5.1 and Appendix G.5 (Table 36). Raw-target results are in Appendix E.1.

Thank you again for reading the code. Both corrections are now part of the released implementation. If they resolve your soundness concerns, we would be grateful if you would reconsider your assessment. If any other part of the code or the paper still looks inconsistent, please tell us and we will check it during the discussion period.

Thank you for the detailed response and for fixing both issues in the released code. The additional analyses address my concerns. I have raised my rating from 2 to 3.

AuthorsOpenReview ↗

Thank You for Your Constructive Feedback

Thank you very much for your thoughtful and constructive feedback. We sincerely appreciate your careful review and valuable suggestions, which have helped us improve our work. We are grateful for your positive reassessment.

Reviewer_6TKfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formulates target-conditioned cellular intervention design as finite-budget ranking: given source and target transcriptional states, a variable library of candidate genetic perturbations, and a budget B, rank the candidates and test the top B. It proposes an evaluation protocol in which candidate utility is the cosine between the held-out replicate response of the (excluded) query perturbation and that of each candidate, computed on batch-disjoint replicate views and summarized by a random- and oracle-normalized, rank-discounted budgeted utility (BU@B). For candidates whose own responses were never measured, it proposes VCDesign-CED: a ridge map from STRING and MAP-KG features to a low-rank response basis, whose predicted effects are scored against the requested transition and fused by per-query z-scoring with a dual-encoder base scorer. On four CRISPRi Perturb-seq contexts, CED improves pooled BU@20 by 0.049 for response-unseen candidates in held-out contexts and BU by 0.032 on held-out K562 identities. Retrieval of measured responses is much better when such responses exist (+0.152), and routing measured candidates to retrieval helps mixed libraries. Controlled analyses suggest that the number of measured rows and the STRING features matter more than context count or richer effect models, and that prediction accuracy does not reliably rank nearby models by design utility.

Critical strengths

S1. The evaluation is decision-aligned and well built: it scores only the executable prefix, grades with batch-disjoint held-out replicates so the same measurements never both form and grade a query, filters queries by replicate reproducibility, and normalizes against random and oracle orderings. This is better practice than most perturbation-model evaluations.

S2. Information regimes are separated explicitly (context transfer versus the candidate's own response masked), with well-designed controls: a size-controlled masking arm, a stratum of candidates the base scorer never trained on, row- and identity-matched atlases, and a preregistered routing comparison.

Critical weaknesses

W1. The novelty claim needs to be revised. The paper states that "to our knowledge, this is the first explicit formulation of target-conditioned cellular intervention design as finite-budget ranking over a variable candidate set". The related-work section covers inverse-design methods (CMap and LINCS, CellNavi, PDGrapher, PAIRING, RevPert, CASCADE, U-Pert) and separates VCDesign from them by output space, access to candidate responses, and evaluation objective, and Section 3.1 cites GeneDisco for the budgeted selection step. The closest prior formulations are not discussed, however. Zhang et al. [2] formulate the design of interventions that move a system toward a desired target mean with few experiments, and apply it to Perturb-CITE-seq to find perturbations that induce a specific cell-state transition. DiscoBAX [3] selects diverse, high-value intervention sets under a budget, which is the paper's own motivation that several distinct useful interventions should be rewarded. Budgeted Perturb-seq design was also studied by IterPert [4] and by BioDiscoveryAgent [5], an LLM agent that selects genes to perturb over rounds. Ranking in-silico perturbations by how well they shift cells toward a target state appears in Geneformer's in-silico treatment analysis [6] and CellOracle [7]. scGPT, which the paper cites only as a forward model, also reports reverse perturbation prediction [8], and earlier cell-conversion work ranks transcription factors toward a target cell type [9]. In my reading, the real contribution is the evaluation protocol and the analysis of information regimes, and the paper should say so and compare its formulation with these works.

[1] Mandi et al. Decision-focused learning: foundations, state of the art, benchmark and future opportunities. JAIR, 2024. [2] Zhang, Cammarata, Squires, Sapsis, Uhler. Active learning for optimal intervention design in causal models. Nature Machine Intelligence, 2023. [3] Lyle et al. DiscoBAX: discovery of optimal intervention sets in genomic experiment design. ICML 2023. [4] Huang, Lopez, Hütter, Kudo, Rios, Regev. Sequential optimal experimental design of perturbation screens guided by multi-modal priors. RECOMB 2024. [5] Roohani et al. BioDiscoveryAgent: an AI agent for designing genetic perturbation experiments. ICLR 2025. [6] Theodoris et al. Transfer learning enables predictions in network biology. Nature, 2023. [7] Kamimoto et al. Dissecting cell identity via network inference and in silico gene perturbation. Nature, 2023. [8] Cui et al. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nature Methods, 2024. [9] Rackham et al. A predictive computational framework for direct reprogramming between human cell types. Nature Genetics, 2016.

W2. What the benchmark measures. Every target state is itself the response to a single CRISPRi knockdown in the same context, and U(q, c) rewards candidates whose response resembles that of the hidden query gene. The task is therefore retrieval of the functional neighbors of an unseen gene, for example members of the same complex or pathway, which are known to share transcriptional phenotypes in these screens [10]. This is consistent with STRING physical interactions carrying most of the transferable signal (dropping STRING costs 0.021 BU@20 while dropping MAP-KG costs 0.0013) and with the authors' own remark that transfer may exploit close interaction partners. It is a different and easier problem than designing toward a state that no single library member produces (disease to healthy, differentiation, drug response), which is the motivation given in the introduction. I suggest (a) a baseline that infers likely query genes from the transition and ranks candidates by STRING proximity to them; (b) gains stratified by whether the query gene belongs to a large co-complex module; and (c) at least one target type not generated by a library member, for example combinatorial targets from Norman et al. [11] with single-gene candidates, or disease versus control signatures.

[10] Replogle et al. Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq. Cell, 2022. [11] Norman et al. Exploring genetic interaction manifolds constructed from rich single-cell phenotypes. Science, 2019.

W3. Effect sizes and practical relevance. Gains are statistically clear but small: +0.049 BU@20 (about 11% relative to the base scorer's 0.440) and about one extra top-5% candidate in 20 picks. The realized-outcome guardrail is null under its preregistered rule, and in the K562 development-fold stress test with a 6,255-candidate vocabulary the BU gain is negative (-0.0103 with raw targets, -0.0250 with unit targets), which the authors acknowledge leaves genome-scale vocabularies outside the established regime. Real screening libraries are genome scale, so the regime where CED helps (pools of 764 to 1,450 candidates) needs to be justified as a realistic design setting.

W4. Baselines. The external methods are degenerate (GEARS predicts a candidate-invariant shift), limited by a small fine-tuning budget (one CellNavi fine-tune per context and class-set configuration, which the authors note reflects that budget rather than the method's attainable performance), or not run (PDGrapher, and STATE [12], which is cited and designed for context transfer). The forward-then-match route uses only the authors' ridge. Several knowledge-based predictors of unseen perturbation effects fit the response-unseen regime directly and should be used as forward models: TxPert [13], which combines multiple knowledge graphs and reports results on the same K562, RPE1, HepG2 and Jurkat screens; PRESAGE [14]; and LLM-derived gene embeddings with a linear head [15]. A non-learned baseline beyond the learned memory branch of Appendix E.2, such as the average measured response of the nearest STRING or MAP-KG neighbors, would also show what the ridge adds.

[12] Adduri et al. Predicting cellular responses to perturbation across diverse contexts with State. Cell, 2026. [13] Wenkel et al. TxPert: leveraging biochemical relationships for out-of-distribution transcriptomic perturbation prediction. arXiv:2505.14919, 2025. [14] Littman et al. Gene-embedding-based prediction and functional evaluation of perturbation expression responses with PRESAGE. bioRxiv, 2025. [15] Chen and Zou. GenePT: a simple but effective foundation model for genes and cells built from ChatGPT. bioRxiv, 2023.

Questions for authors

Q1. How does a baseline that takes the base scorer's top identities and then ranks their STRING neighbors perform? Q2. Are the gains concentrated in queries whose gene belongs to a large complex (ribosome, Mediator, proteasome)? Q3. How does the method behave when the target is not produced by any single library member? Q4. How sensitive are the main results to the base-scorer training seed and the masking-fold hash? Q5. Candidates with no knowledge modality score exactly zero in both branches, and Appendix D.7 reports that their share of top-10 picks falls from 0.0035 to 0.0016. Where do they sit in the full ranking after per-query z-scoring, and does this help or hurt BU?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 6TKf (1/3)

Thank you for the detailed review and concrete suggestions. We ran all your suggestions except TxPert and PDGrapher, for reasons given under W4. In summary:

  • W1, novelty: revised as you suggest. Section 1 now states the VCDesign framework as the first contribution, and Section 2 cites each work you name.
  • W2, what the benchmark measures: for single-gene targets, in large part yes: about half of the gain depends on measured functional neighbours, and the paper now says so. The gain also holds on two target types no library member produces, 39 Hallmark programmes (+0.066) and 514 disease-versus-normal signatures (+0.037), and your query-inference plus STRING-proximity baseline does not reproduce it.
  • W3, effect sizes: the pools are the eligible libraries of the three held-out screens. The revision claims that regime and states the K562 stress test as a limit.
  • W4, baselines: we added six forward-then-match routes and STATE. A non-learned STRING-neighbour average (STRING-kNN in the paper) beats CED, and the paper says so. PRESAGE retrained under our masking does not beat CED.
  • Q4: with a second masking hash and two more base-scorer seeds, the gain stays between +0.0492 and +0.0525.
  • Q5: on K562, knowledge-free candidates sit near the top of the base scorer's ranking, which hurts it, and fusion moves them down. The headline contains none.

Numbers are BU@20 (higher is better; 0 for random, 1 for the best ordering) on candidates whose own responses are masked from every training context, pooled over RPE1, HepG2 and Jurkat. Brackets are 95% cluster-bootstrap intervals over query genes, Hallmark sets or signature clusters. New results are development evidence with one base-scorer seed unless stated.

W1. "The novelty claim needs to be revised."

We agree, and the revision follows your reading. The "first explicit formulation" sentence is gone. Section 1 states three contributions: (1) the VCDesign framework, which scores the first BB of a ranking over a variable library by measured held-out responses, on one scale for retrieval, identification, forward simulation and direct scoring, (2) ranking candidates with no measured response from related genes' measured responses, with CED and a STRING-neighbour average side by side, and (3) the dependence of design quality on the functional coverage of measured perturbations, with the finding that better response prediction does not ensure better selection. Section 2 now cites each work you name. How our setting differs:

  • Budgeted design (Zhang et al. 2023, DiscoBAX, IterPert, BioDiscoveryAgent) acquires data round by round within one screen. We study one-shot ranking of a library toward a transcriptomic target, transfer to new cell lines and unmeasured candidates, and grading of the chosen prefix with held-out replicates.
  • Target-state ranking: Geneformer's in-silico treatment and CellOracle simulate, then compare: the forward-then-match route our framework evaluates (W4). scGPT's reverse perturbation prediction is scored on recovering the perturbation behind a state, and Mogrify (Rackham et al. 2016) checks predicted transcription factors against known conversions. We grade the budgeted top-B by measured outcomes instead.
  • Decision-focused learning (Mandi et al. 2024) is now cited for the gap between prediction and decision quality that Section 5.4 tests.

W2. "The task is therefore retrieval of the functional neighbors of an unseen gene"

For single-gene targets, in large part yes: these targets reward candidates that resemble the hidden query gene, and about half of the gain depends on measured neighbours, as the paper now states. Beyond them, the gain holds on targets with no hidden query gene (c), and proximity alone does not reproduce it (a). The introduction now names the three target types we test (single-gene, programme, disease-signature) rather than motivating the task by disease-to-healthy or differentiation design (Sections 1 and 6).

  • Community masking (preregistered): when a median 60% of a candidate's 20 nearest STRING neighbours are also unmeasured, about half of each method's gain survives (retention 0.569 [0.508, 0.645] for CED, 0.518 [0.461, 0.574] for the STRING-neighbour average), and the neighbour average stays ahead (+0.0055 [+0.0024, +0.0091]). Neither preregistered hypothesis held as defined (Section 5.3, Fig. 3d, Appendix G.4).
  • (a) Query inference plus STRING proximity: taking the base scorer's top mm candidates as the inferred query genes and ranking by STRING cosine to them, with no response used, changes fused BU@20 by -0.0588 (m=1m = 1), -0.0359 (m=5m = 5) and +0.0122 (m=20m = 20). At m=20m = 20 it stays -0.0335 [-0.0383, -0.0278] below VCDesign-CED (Appendix G.6). On programme targets, proximity to the programme's genes scores at chance. What transfers is the measured response of neighbours, not proximity. We tested one inference rule only.
AuthorsOpenReview ↗

Response to Reviewer 6TKf (2/3)

W2, continued.

  • (b) Large co-complex modules: the gain is larger there but not concentrated there. With Complex Portal and a threshold of at least 10 genes fixed in advance, CED minus BASE is +0.0600 [+0.0545, +0.0670] for 302 query genes in large complexes, +0.0432 [+0.0348, +0.0496] for 289 in smaller ones, and +0.0443 [+0.0386, +0.0510] for the 385 in no annotated complex, which keep about 90% of the pooled gain. The STRING-neighbour average's edge over CED does not differ between strata (+0.0027 [-0.0041, +0.0097]) (Appendix G.6).
  • (c) Targets not produced by any library member: the gain holds on both types we added (Section 5.2, Fig. 2c,d, Appendix G.1).
    • 39 MSigDB Hallmark programmes, both directions, inclusion fixed on K562 alone: BASE+CED minus BASE is +0.066 [+0.042, +0.087], positive in every held context, and [+0.031, +0.080] when the 12 correlated programme groups are resampled together. Two of six context ×\times direction cells include 0.
    • 514 disease-versus-normal GEO signatures (Wang et al., Nat. Commun. 2016; preregistered), each used as a reversal and a mimic target: +0.037 [+0.030, +0.044], positive in every context and about equal in both directions. The STRING-neighbour average leads CED here (+0.012 [+0.009, +0.015], not in HepG2), while on programmes the two show no detectable difference.
    • Measurement stays far ahead: the candidate's own held-out response reaches 0.697 on programmes and 0.755 on signatures, against at most 0.179 and 0.214 for any predictor.

Scope. We did not use Norman et al. combinatorial targets, which come from a CRISPRa screen not protocol-matched to these CRISPRi screens (Appendix F). Disease signatures come from tissues, so this result concerns the direction of the response, as in connectivity mapping, not reaching a healthy tissue state in a cell line.

W3. "Real screening libraries are genome scale, so the regime where CED helps (pools of 764 to 1,450 candidates) needs to be justified as a realistic design setting."

The pools of 764 to 1,450 candidates are the eligible libraries of the three held-out screens: every usable perturbation that a training context measured and that has knowledge features (Table 2, Appendix D.9). Each screen targets 2,393 genes, so the revision claims that regime and not genome scale. We agree the gains are moderate (Section 6, Appendices F and D.10).

  • Size: +0.0492 [+0.0456, +0.0534] over a base level of 0.440, about 11%, and +0.0668 [+0.0616, +0.0709] for the STRING-neighbour average. In the first 20 picks VCDesign-CED holds on average 6.92 of the pool's top-5% candidates, against 5.81 for the base scorer and 1.06 at random. The gain is +0.066 on programmes and +0.037 on disease signatures, and stable across seeds and hashes (Q4).
  • Genome scale: in the K562 stress test the gain falls beyond 1,000 candidates and is not positive at 6,255 (-0.0103 [-0.0154, -0.0048] BU‾\overline{\mathrm{BU}} with raw targets, -0.0250 with unit targets, -0.0026 with per-subset normalization; Appendices F and G.5). Section 6 and Appendix F state this limit.
  • Realized-outcome guardrail: null in median (PHR@20 +0.0171 [+0.0083, +0.0258] in mean, 0.000 in median), and Section 6 states it as a limit.

W4. "The forward-then-match route uses only the authors' ridge."

We added six forward-then-match routes and ran STATE. CED is not the strongest solver, and the paper now says so: Table 1 shows BASE+STRING-neighbour average as the best response-unseen entry on single-gene targets (Section 5.1, Table 1, Sections 5.3 and 5.4, Appendices D.6, G.2, G.3). Single-gene targets, response-unseen candidates, each route fused with the base scorer:

RouteMinus VCDesign-CED
STRING-neighbour average, every kk from 1 to 200+0.0036 [+0.0014, +0.0056] to +0.027
GenePT neighbour average, kk = 10 to 50no detectable difference
MAP-KG neighbour averagebelow at every kk
GenePT linear head-0.0235 [-0.0267, -0.0212]
PRESAGE, 35 static sources-0.0098 [-0.0124, -0.0071]
PRESAGE, plus fitness and imaging screens+0.0013 [-0.0007, +0.0036]
  • STRING-neighbour average (your non-learned baseline): on programmes the two show no detectable difference (+0.002 [-0.005, +0.015]). The ridge fits measured responses better than the k=20k = 20 average in every neighbour-support stratum, yet ranks worse.
  • MAP-KG and GenePT: all beat the base scorer, and none beats the STRING average at any kk.
AuthorsOpenReview ↗

Response to Reviewer 6TKf (3/3)

W4, continued.

  • Matched-capacity forward MLP: it fits held-out responses as well as the ridge, but its predictions collapse toward the shared response. Fused with the base scorer it adds +0.0009 [-0.0005, +0.0021], a null, and stays -0.0504 [-0.0534, -0.0469] below VCDesign-CED (alone: -0.330 [-0.344, -0.309], the preregistered contrast). Two seeds.
  • PRESAGE: retrained per held context and masking fold on exactly CED's training rows, at the released K562 genome-wide configuration, two seeds. The static arm is below VCDesign-CED in every context and seed, both arms are below the STRING average, and its predictions do not collapse.
  • STATE (ST-HVG-Replogle zero-shot checkpoints): these train on the other cell lines' measured responses to the same perturbations, so we ran it where candidates are measured. Fused with the base scorer it reaches 0.545, above VCDesign-CED in the measured setting (0.508; +0.026 [+0.021, +0.031]) and below profile retrieval (0.675). A query-shuffle control removes the gain.
  • Not run: TxPert, whose public release supports inference only, from K562-trained checkpoints, so it cannot be retrained with candidates masked (the models of its paper also use two proprietary graphs). ST-SE, whose embedder's pretraining corpus cannot be shown to exclude the held cell lines. PDGrapher, with 45% of its PPI nodes on the shared gene axis and a genetic mode that needs a mutation vector RPE1 lacks. GEARS and CellNavi stay as labelled companions with the limits you note.

Q1. "How does a baseline that takes the base scorer's top identities and then ranks their STRING neighbors perform?"

Worse than VCDesign-CED at every setting: -0.0335 [-0.0383, -0.0278] below it with the top 20 identities, and below the base scorer itself with the top 1 or 5 (W2a).

Q2. "Are the gains concentrated in queries whose gene belongs to a large complex (ribosome, Mediator, proteasome)?"

Larger there, but not concentrated: +0.0600 in large complexes against +0.0443 for genes in no complex, which keep about 90% of the pooled gain. Named families (descriptive): ribosome +0.071 (93 genes), Mediator +0.050 (8), proteasome +0.108 (17) (W2b).

Q3. "How does the method behave when the target is not produced by any single library member?"

The gain holds: +0.066 on 39 Hallmark programmes and +0.037 on 514 disease signatures, with measurement far ahead. On programmes, CED alone and the fused method show no detectable difference (+0.028 [-0.014, +0.077], CED alone minus fused). On disease signatures the fused method is better by 0.020 [0.009, 0.031] (W2c).

Q4. "How sensitive are the main results to the base-scorer training seed and the masking-fold hash?"

Little. Against the published +0.0492 [+0.0456, +0.0534], a second masking hash gives +0.0506 [+0.0462, +0.0543], with fold agreement 0.20 to 0.22, chance for five folds, so every masked fit was refitted on a new partition. Two more base-scorer seeds give +0.0525 [+0.0494, +0.0559] and +0.0500 [+0.0458, +0.0539], and +0.061 to +0.066 on programmes and +0.033 to +0.042 on disease signatures. The ridge penalty is 10410^4 in every refit. Three seeds and two hashes give a range, not a seed-variance estimate (Appendix G.5).

Q5. "Where do they sit in the full ranking after per-query z-scoring, and does this help or hurt BU?"

On the K562 split they sit near the top of the base scorer's ranking (median rank percentile 0.07), which hurts its BU. Fusion moves them down (median 0.25 to 0.26), which is the fall in top-10 share you cite.

  • Why: the base scorer gives all 97 of them exactly 0, above the other candidates' median raw score (-2.94), so they fill a third of its top 50, although their measured utility is unremarkable (percentile 0.45 to 0.47). Moving them to the bottom raises its BU‾\overline{\mathrm{BU}} by 0.0064 [0.0022, 0.0101].
  • Effect on the K562 gain: about half of CED's K562 BU‾\overline{\mathrm{BU}} gain is this correction (+0.0319 as reported, +0.0148 [+0.0108, +0.0199] with them demoted). The gains at B = 10 and 20 are essentially unchanged.
  • Held-out contexts: the headline (+0.0492) contains none of them, because every eligible held-context candidate has at least one knowledge modality.

The paper reports the demoted gain as a sensitivity analysis next to the K562 result (Section 5.2, Appendix G.6, Fig. 9). The rank percentiles and the base-scorer correction are given here only.

Your suggestions reshaped the framing, the baselines and the target types of the paper. If the revision answers your concerns, we would appreciate it if you would consider updating your assessment. If a concern still holds you back, please let us know which one, and we will respond during the discussion period.

Reviewer_EwTdOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new evaluation pipeline for cellular pertubation prediction. Instead of doing a "forward" prediction where a model ist tasked to predict the effect of a perturbation on a control cell, this paper proposes to predict genes whose perturbation will correspond to a given perturbation profile. The authors then propose a new model, VCDesign-CED, that achieves the best performance on this task. The paper then studies the impact of different data ablation strategies on performance (holding out contexts and genes).

Critical strengths
  • The problem formulation is interesting, relevant, and practical.
  • The experimental validation is extensive, and the authors distinguish between generalizing to a new cell line versus generalizing to a new candidate without historical perturbation measurement.
Critical weaknesses

The problem formulation is not as novel as the paper suggests. CMap already considered finding genes that match a given perturbation response. The authors should clarify why their approach is significantly novel in that respect.

Is seems that s_base and s_eff are trained separately although it's not clearly stated in the paper. This is an important part of the paper that is currently not given enough details in the main text. The authors should motivate more clearly why such an additive setup makes sense in the methods section, even if the experimental results show it improves over s_base alone.

One important missing baseline in my opinion is model that directly predicts rcr_c from xsx_s and kck_c with the same capacity as the encoder maps in Section 4.2. This would allow to check whether forward prediction + ranking would actually match the performance of your approach.

The generalization of the results is questionable due to the random nature of the gene masking. Masking test genes at random leaves highly correlated genes in the training data that can easily be identified using biological databases such as STRING (used in k_c). Could the authors consider a stratified split where similar STRING genes are masked jointly ?

There is relatively little machine learning contribution. Little to no detail is given to the modeling strategy, architecture, and design. As a reader, I get little practical insights as to what are the necessary ingredients to perform well at this perturbation retrieval task.

Questions for authors
  • Can the authors motivate more clearly their contribution in terms of problem formulation compared to CMap ?
  • Can the authors describe more clearly the training strategy of the model ?
  • Can the authors include a baseline model that directly predicts rcr_c from xsx_s and kck_c with the same capacity as the encoder maps in Section 4.2.
  • Could the authors use a stratified split where similar STRING genes are masked jointly ?
5 comments
AuthorsOpenReview ↗

Response to Reviewer EwTd (1/3)

Thank you for the careful review. We ran each analysis you asked for. In summary:

  • W1, CMap: we credit CMap with ranking interventions toward a target from measured profiles, and no longer claim that as new. What CMap does not cover, and the paper does: the VCDesign framework, which grades the chosen top-B by measured outcomes on one scale for every solver family, and candidates with no measured response.
  • W2, training: yes, the two branches are trained separately. Section 4 now states the training procedure and why the sum is additive.
  • W3, forward baseline: added. A matched-capacity forward model does not match VCDesign-CED (-0.330), although it fits held-out responses as well, and PRESAGE retrained under our masking does not beat it.
  • W4, stratified split: run under a preregistered protocol. About half of each method's gain survives when whole STRING communities are masked jointly.
  • W5, ML contribution and insights: Sections 5.3 and 5.4 give the evidence for what it takes to rank candidates well (list under W5). Section 4 and Appendix B give the full model.

Numbers are BU@20 (higher is better; 0 for random, 1 for the best ordering) on candidates whose own responses are masked from every training context unless marked measured, pooled over the three held-out cell lines. Brackets are 95% cluster-bootstrap intervals over query genes (Hallmark programmes or signature clusters for those targets). New results are development evidence (one base-scorer seed for most analyses, two for the forward MLP and PRESAGE). W1 to W5 label your five weakness paragraphs in order.

W1. "CMap already considered finding genes that match a given perturbation response."

We agree. Matching a target signature against measured perturbation profiles is CMap's idea, and Sections 1 and 2 credit it. The contribution is restated as (1) the VCDesign framework, which scores the first BB of a ranking over a variable library by measured held-out responses, on one scale for retrieval, identification, forward simulation and direct scoring, (2) evidence that candidates with no measured response can be ranked from the measured responses of related genes, and (3) evidence that design quality depends on the functional coverage of measured perturbations, and that better response prediction does not ensure better selection. Three points separate this from CMap:

  • Unmeasured candidates: CMap can score a candidate only if its profile was measured. Our main setting is the candidate whose own response is absent from every training context.
  • Evaluation: CMap is judged by connectivity or identity recovery. We grade the chosen prefix with replicate outcomes not used to form the query, so several distinct interventions can earn credit.
  • CMap as a solver: CMap-style retrieval is one of our solvers. When a candidate was measured in other contexts it is the best solver we tested: 0.675 against 0.508 for VCDesign-CED, a paired advantage of +0.1519 [+0.1424, +0.1602] (Section 5.3, Table 1).

W2. "Is seems that sbases_{\text{base}} and seffs_{\text{eff}} are trained separately although it's not clearly stated in the paper."

Yes, they are trained separately, and Section 4 now says so in the main text, with the reason for the additive form.

  • Base scorer: a dual encoder trained once on K562 training identities with cross-view utility labels, a replicate-informed pairwise loss (epochs 1 to 5) and an added top-20 set loss (epochs 6 to 10). The epoch is chosen by median BU‾\overline{\mathrm{BU}} on the K562 development split. It is not retrained for any held context or masking fold.
  • CED: a closed-form ridge from STRING and MAP-KG features to a 256-dimensional SVD basis of training responses, with the penalty chosen by cross-validation grouped by gene. In a held-out context it is refitted on the other three contexts, once per masking fold.
  • Fusion: each branch is z-scored per query over the context's candidate pool, and the two are added with weight one. No fusion weight is fitted; the form and unit weight were chosen on the K562 development split.
  • Why additive and separate: the branches use different information. The base scorer learns query-candidate compatibility from K562 utility labels, and CED transfers measured responses of other genes. Training them apart lets CED be refitted for every held context and masking fold while the base scorer stays fixed, so the masking experiments change only the effect branch, and the equal-weight sum has no free weight to tune on evaluation pools.
  • When the sum helps: on single-gene targets and disease signatures fusion improves on CED alone (by 0.020 [0.009, 0.031] on signatures). On programme targets the base scorer carries little signal, and CED alone and the fused method show no detectable difference (Section 5.2, Appendix G.1). Appendix B gives the full architecture, losses and hyperparameters.
AuthorsOpenReview ↗

Response to Reviewer EwTd (2/3)

W3. "One important missing baseline in my opinion is model that directly predicts rcr_c from xsx_s and kck_c with the same capacity as the encoder maps in Section 4.2."

We added it, and forward prediction plus ranking does not match VCDesign-CED (Table 1, Section 5.4, Fig. 4b, Appendix G.3).

  • The model: an MLP mapping (xs,kc)(x_s, k_c) to rcr_c with the layout of the encoder maps in Section 4.2, 24.9M parameters against 30.1M (the difference is the gene-axis layers, 6,550 genes against 8,248). It trains on exactly CED's training rows for each masking fold, stops early on a gene-grouped 10% of those rows, and ranks candidates by the cosine between the predicted response and the requested transition. Two seeds.
  • Result: alone it is -0.330 [-0.344, -0.309] below VCDesign-CED and -0.189 [-0.204, -0.173] below CED alone, and added to the base scorer it changes BU@20 by +0.0009 [-0.0005, +0.0021], a null.
  • Why: it fits held-out responses as well as or better than the ridge (validation cosine 0.139 to 0.167 against 0.136 to 0.154), but its predictions collapse toward the response shared by all knockdowns. The mean cosine of a prediction to the mean prediction is 0.90 to 0.96, against 0.74 to 0.78 for the ridge. Training on targets centred on the mean response (post hoc) narrows the gap to CED alone to -0.125 [-0.144, -0.108], still below the ridge.
  • PRESAGE, a published knowledge-based forward model retrained under our masking with two seeds, does not collapse (0.67 to 0.77) but, fused with the base scorer, does not beat VCDesign-CED: -0.0098 [-0.0124, -0.0071] with its static knowledge sources, and no detectable difference with its fitness and imaging screen sources added (+0.0013 [-0.0007, +0.0036]) (Appendix D.6).
  • STATE (ST-HVG-Replogle zero-shot checkpoints) trains on the other contexts' measured responses to the same perturbations, so we ran it where candidates are measured. There, fused with the base scorer, it reaches 0.545 against 0.508 for VCDesign-CED (+0.026 [+0.021, +0.031]) and 0.675 for profile retrieval (Appendix D.6).

Scope. One architecture and training recipe with two seeds, so this does not show that forward models in general are unsuitable for design. A second forward-then-rank route, the non-learned STRING-neighbour average in W5 (STRING-kNN in the paper), does beat CED.

W4. "Could the authors consider a stratified split where similar STRING genes are masked jointly ?"

Yes, under a preregistered protocol. About half of each method's gain survives, so the gain does not vanish when a candidate's STRING community is masked with it, but about half of it depends on measured neighbours (Section 5.3, Fig. 3d, Appendix G.4).

  • Design: communities from the STRING physical graph, found without response data, are masked jointly: every gene in a fold loses its responses in every training context. A control assigns random genes to folds of identical size. Under community masking a candidate keeps a median 40% of its 20 nearest STRING neighbours in the atlas, against 80% under the control.
  • Share of the gain that survives, control to community masking: CED 0.569 [0.508, 0.645] (+0.0473 to +0.0269), STRING-neighbour average (k=20k = 20) 0.518 [0.461, 0.574] (+0.0690 to +0.0358).
  • Preregistered contrast, difference in drops (CED minus neighbour average): -0.0134 [-0.0166, -0.0106]. CED loses less in absolute terms, from a smaller starting gain, and the neighbour average still beats it under community masking (+0.0055 [+0.0024, +0.0091]).
  • Verdict: neither the preregistered local hypothesis nor the global one (longer-range transfer by CED) held as defined. Because 40% of the nearest neighbours stay measured, the surviving half cannot be split between those neighbours and longer-range structure.

Under the standard masking, the gain also holds on targets that are not any library member's response: +0.066 [+0.042, +0.087] on Hallmark programmes and +0.037 [+0.030, +0.044] on disease-versus-normal signatures (Section 5.2, Appendix G.1).

AuthorsOpenReview ↗

Response to Reviewer EwTd (3/3)

W5. "There is relatively little machine learning contribution. Little to no detail is given to the modeling strategy, architecture, and design. As a reader, I get little practical insights as to what are the necessary ingredients to perform well at this perturbation retrieval task."

The contribution is the VCDesign framework and what it shows about the ingredients, not a new architecture. Because the framework scores every solver family by the outcomes of the candidates it selects, each ingredient becomes a controlled test. The evidence in Sections 5.3 and 5.4 supports the conclusion of Section 6: design quality depends on the model together with the measurements it can draw on.

The ingredients we tested:

  1. Use the measurement when it exists. Retrieving a candidate's response measured in another cell line beats every predictor, by +0.1519 [+0.1424, +0.1602] over VCDesign-CED. With half the library response-unseen, routing measured candidates to retrieval and the rest to effect transfer adds +0.0125 [+0.0100, +0.0150] over CED for all.
  2. More measured evidence moves the gain more than a better effect map. Atlases with all rows of one, two and three training contexts gain +0.025, +0.042 and +0.049 over the base scorer, while refinements of the ridge gain at most +0.0009 [+0.0006, +0.0013]. STRING carries most of the signal: removing it lowers BU@20 by 0.0211 [0.0177, 0.0235], against 0.0013 for MAP-KG. Masking each candidate's STRING community roughly halves the gain (W4).
  3. Match how local the transfer is to the target. On single-gene targets the non-learned STRING-neighbour average beats our ridge at every kk from 1 to 200 (+0.004 to +0.027, fused). On Hallmark programmes close neighbours lose their edge (k=1k = 1: -0.017), only broad averages lead (k=200k = 200: +0.034, effect alone), and at k=20k = 20 the two show no detectable difference. No MAP-KG or GenePT (LLM) variant beats the STRING average at any kk (Appendix G.2).
  4. Select the estimator by decisions, not by response fit. Within the neighbour average, fit to held-out responses rises with kk (median cosine 0.040 to 0.232) while effect-only BU@20 falls (0.437 to 0.370), so a fit criterion picks the worst kk for design. The ridge fits better than the k=20k = 20 average yet ranks worse, and the forward MLP fits as well as the ridge yet ranks far worse (W3). Fit does track large changes in the available data (Spearman ρ\rho = 0.31 to 0.71 across atlas configurations), but it did not choose among the estimator variants we tested (Section 5.4, Fig. 4).
  5. Fuse with a direct scorer when it carries target signal. Fusion helps on single-gene and disease targets. On programme targets the base scorer carries little signal, and CED alone and fusion show no detectable difference (W2).

Modeling detail. Section 4 now gives the training strategy (W2). Appendix B lists every layer, loss and hyperparameter, and Appendices G.2 and G.3 add the new baselines.

Q1. "Can the authors motivate more clearly their contribution in terms of problem formulation compared to CMap ?"

Yes, see W1. CMap matches measured profiles and is judged by connectivity or identity recovery. VCDesign puts CMap-style retrieval on one scale with the other solver families, grades the chosen top-B by measured outcomes, and also ranks candidates with no measured profile. Where profiles exist, retrieval is our best solver (Sections 1 and 2, Table 1).

Q2. "Can the authors describe more clearly the training strategy of the model ?"

Yes, see W2. The two branches are trained separately: the base scorer once on K562, CED as a closed-form ridge refitted for each held context and masking fold. They are added as per-query z-scores with weight one, chosen on the K562 development split (Section 4, Appendix B).

Q3. "Can the authors include a baseline model that directly predicts rcr_c from xsx_s and kck_c with the same capacity as the encoder maps in Section 4.2."

Included, see W3. Forward prediction plus ranking does not match our approach: the matched-capacity model fits held-out responses as well as CED, but fused with the base scorer it adds nothing detectable and is -0.0504 [-0.0534, -0.0469] below VCDesign-CED (alone: -0.330; Table 1, Section 5.4, Appendix G.3).

Q4. "Could the authors use a stratified split where similar STRING genes are masked jointly ?"

Yes, see W4. About half of each method's gain survives preregistered STRING-community masking (CED 0.569 [0.508, 0.645], STRING-neighbour average 0.518 [0.461, 0.574]; Section 5.3, Fig. 3d, Appendix G.4).

Thank you again for the suggestions. The forward baseline, the stratified split and the framework statement are now central to the paper. If they answer your concerns, we would be grateful if you would reconsider your score. If something still stands in the way, please tell us, and we will address it during the discussion period.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank all three reviewers, whose constructive reviews substantially improved the paper. Detailed replies follow under each review.

Main message. Design quality depends on both the model and the measurements it can draw on, and it should be judged by the experiments a method selects, not by how accurately it predicts responses.

Common ground. The reviewers describe the problem formulation as "interesting, relevant, and practical" (EwTd) and the evaluation as "better practice than most perturbation-model evaluations" (6TKf), and note that the framework "guides method selection across different information regimes" (8aDp).

What the paper shows. VCDesign asks a planning question: given a desired cell state and a library of candidate knockdowns, which few should a laboratory test? Every method that ranks the library, whether by retrieval, identification, forward simulation or direct scoring, is scored in the same way: by the responses of its top BB choices measured in held-out replicates (BU@B, from 0 for random choice to 1 for the best choice). On this common scale, the best method depends on what has been measured: where a candidate's response was measured in other cell lines, retrieving it outperforms every predictor (+0.152 BU@20 over VCDesign-CED). For candidates whose own responses are withheld from effect learning (response-unseen), CRISPRi screens of three held-out cell lines support three conclusions:

  • (a) Response-unseen candidates can still be selected well by transferring the measured responses of functionally related genes. BU@20 improves from 0.440 with direct scoring to 0.499 with our learned map (VCDesign-CED) and to 0.530 with a simple average over STRING neighbours (STRING-kNN).
  • (b) A major source of this gain is the measured responses of related genes. Masking each candidate's STRING community with it roughly halves the gain on single-gene targets (preregistered, EwTd W4, Section 5.3).
  • (c) Better response prediction does not guarantee better selection. As STRING-kNN averages over more neighbours, its fit to held-out responses improves (median cosine 0.040 to 0.232) while its BU@20 without the base scorer declines (0.437 to 0.370).

How the revision addresses each concern. Gains are BU@20 improvements over the base scorer (direct scoring) unless stated, with 95% cluster-bootstrap intervals.

  1. The novelty claim now rests on the framework (6TKf W1; EwTd W1, W5). We agree that ranking interventions toward a target state is not new, since CMap does this for measured profiles, and we removed that claim. The contribution is the VCDesign framework, which scores the selected top BB by measured outcomes on one scale for all solver families, together with conclusions (a) to (c) for candidates that CMap cannot score. Related Work now covers CMap, budgeted design and target-state ranking (Sections 1 and 2).

  2. Both implementation issues found by 8aDp are fixed, and the headline result is stable (6TKf Q4). Cross-validation now groups folds by gene and refits the response basis within each fold. The selected penalty is unchanged, so no BU@20 value changes and the headline paired gain remains +0.0492 [+0.0456, +0.0534]. Eq. 5 now describes the normalization the code uses, and the one in the submitted text gives +0.0548. Choosing the training target without access to the held-out context gives the same choice. A second masking assignment and two more base-scorer seeds give +0.0500 to +0.0525 (Sections 4 and 5.1, Appendix G.5).

  3. The gain extends to targets that no single gene produces (6TKf W2, Q3). It is +0.066 [+0.042, +0.087] on 39 Hallmark programmes (robust to resampling correlated programmes together) and +0.037 [+0.030, +0.044] on 514 disease-versus-normal signatures (Section 5.2).

  4. With the requested baselines added, our learned map is not the strongest solver (EwTd W3; 6TKf W4). The non-learned STRING-kNN outperforms VCDesign-CED on single-gene targets at every kk (+0.0036 to +0.027) and on disease signatures (+0.012). Learned forward models do not. Fused with the base scorer, a matched-capacity forward MLP is -0.050 [-0.053, -0.047] below VCDesign-CED although it fits responses equally well, and PRESAGE retrained under our masking does not exceed it. STATE, evaluated in the measured setting because it uses the candidates' responses from other cell lines, is above VCDesign-CED when fused with the base scorer (+0.026) but below profile retrieval. TxPert and ST-SE were not run, as explained in our reply to 6TKf (Table 1, Sections 5.2 to 5.4).

  5. Limitations are stated explicitly (Section 6). Measured responses remain far ahead of any prediction (0.697 against at most 0.18 on programme targets), the gain shrinks toward genome-scale libraries, and the realized-outcome check shows no median gain. The results cover single-gene CRISPRi in four cell lines, and most new analyses use one base-scorer seed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.