- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
COHERE is a few-shot perturbation predictor. It predicts the response to perturbation (p) in a target context (c) when same-perturbation responses is seen in other contexts and the new context is also seen with other perturbations. The process is as followed: (i) Each source response is summarized per gene by a mean change, a residual occurrence logit, and five positive-expression quantiles. These are Huber-aggregated across source contexts and projected onto a 128-dimensional basis from an SVD of training mean responses. (ii) A per-context low-rank affine map, shared across the three descriptor families and fitted on the target training panel, adapts the basis coordinates. Only the coordinate change is back-projected, so source components outside the basis are retained. (iii) A residual from ESM2 or Morgan-fingerprint features corrects the mean. A deterministic realization step then converts mean, occurrence, and quantiles into N non-negative cells that conserve the predicted gene-wise mean, with values assigned to control anchors by a pretrained ordering network.
The evaluation covers six benchmarks (PBMC cytokines, Replogle-Nadig CRISPRi, SciPlex3, Frangieh21, OP3, McFaline) with 13 Cell-Eval metrics against grouped means, CPA, CellFlow, scGen, scPILOT, TxPert, STATE, and PerturbDiff. COHERE has the best point estimate on 11 of 13 metrics on PBMC and 9 of 13 on Nadig, and the best AUPRC on three of four external datasets. Fixed-mean interventions show that replacing source with learned shape descriptors raises AUPRC on all six datasets, driven by the occurrence channel. Matched retraining shows that source-preserving reconstruction beats subspace-only reconstruction in DE recovery and marginal Wasserstein error on PBMC and Nadig.
Critical strengths
-
The fixed-mean factorial design isolates what marginal shape contributes to DE recovery. Because realization conserves the predicted mean exactly, swapping occurrence and quantile descriptors at fixed means separates shape effects from mean effects, which most distributional methods cannot do. On Nadig, learned occurrence adds ~0.08 AUPRC. The effect survives raw-q and unadjusted-p rankings, and matched retraining with mean-only adaptation reproduces it. The equal-positive control shows that DE ranking and marginal fidelity can move in opposite directions. This matters for my assessment because it is the paper's most defensible finding, and it is useful to anyone building or interpreting Wilcoxon-based perturbation benchmarks.
-
The reporting is transparent. The appendix documents results that weaken the main story: negative transfer on PBMC, score saturation in baseline DE metrics, STATE's lower Nadig W1, and the sensitivity of the PBMC shape effect to score clipping. This makes the claims auditable, and most of my concerns below come from the paper's own tables.
Critical weaknesses
-
The titular claim does not survive the paper's own simplest strong baseline, and the DE comparison is confounded by cell realization. Direct source transfer (source descriptors plus COHERE's realization, no adaptation) appears only as an internal control. On PBMC it beats COHERE and every listed baseline on DEOver, DEPrec, LFCSpear, AUPRC, and AUROC. Including it would cut COHERE's PBMC wins from 11 to at most 8 of 13 and remove its DE advantage there. Direct transfer also has lower MSE than COHERE on OP3 and Frangieh21 and higher AUPRC on McFaline.
-
Separately, Table 19 shows eight PBMC baselines at the label prevalence because most of their genes hit the 1e-10 FDR floor, so their DE scores reflect how each method materializes cells, not what it predicts. Much of COHERE's DE advantage may therefore come from realization and source evidence rather than learned adaptation.
-
The mean-response claim is also mixed: COHERE's MSE is worse than a grouped-mean baseline on SciPlex3 and OP3, and worse than four baselines on Frangieh21. This is consequential because the abstract's "broad performance advantages" in mean prediction, discrimination, and DE recovery is the paper's main empirical claim. Also, SciPlex3 is tested but not Tahoe-100M.
-
Component claims are based on single-seed fits and small effects, and several components contribute nothing measurable.
Questions for authors
-
Realization control: If the pseudobulk means from STATE, PerturbDiff, and mean/perturbation are passed through COHERE's realization with source shape, how do their DE metrics compare with COHERE's? This would separate gains from adaptation and gains from realization.
-
Seed variance: Please report at least three training seeds for COHERE (and ideally STATE) on PBMC and Nadig, with the seed standard deviation for the Table 1 headline metrics and the Table 2 contrasts.
-
Baseline fidelity: STATE was retrained from scratch with an adapted architecture, and PerturbDiff uses released checkpoints. Do the PBMC and Nadig numbers match those reported for these models on the same partitions in the PerturbDiff comparison?
-
Minimal method: Given that the identity residual and allocator have no measurable effect on any reported metric, which components do you consider essential? What does a minimal version (mean plus occurrence adaptation, residual skip, realization) score in Table 1?
-
PBMC DE target: Adaptation lowers pooled PBMC AUPRC but raises within-context AP, and H.2 attributes the pooled loss to new false positives among non-DE genes. Which DE target do you consider correct for PBMC, and why should the pooled protocol be primary?
Thank the authors for their detailed response and the large number of additional experiments. I have read the revised paper, the authors' response, and the comments from other reviewers. Here is my feedback:
Overall, I like this simple and effective method. It separates the transfer of the mean and the distribution shape, and then unifies them using a mean-fixed realization. The idea is clear. The fixed-mean experiment cleanly attributes the gains on DE to the shape, which current distributional models cannot do. However, I still have a few concerns:
I believe the greatest value of this method is that it can act as a plug-in. Like the authors' hybrid experiments ( A.2 in Reviewer AZtC), the realization and fixed-mean analysis do not depend on COHERE itself and can be attached to any mean prediction model. A very natural combination would be: a model with good mean prediction (like STATE) plus a mean-zero proportion curve fitted by the target control, plus the realization. Such combination does not need a panel, and it holds strong promise for achieving both a good mean and a good DE shape. This could be a useful reference for future cross-context zero-shot methods.
Therefore, I believe this paper has value, and I am maintaining my score.