- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes the AdaCell framework, which keeps the diffusion model frozen and, instead, employs a population-level reward mechanism at test time to steer the generation of cell populations toward more biologically plausible responses. Overall, the paper is well-motivated, offers a novel perspective, and is supported by extensive experiments. However, there remains clear room for improvement regarding the rigor of its theoretical justification, the fairness of certain experimental designs, and the boundaries of its core assumptions.
Critical strengths
- The paper accurately identifies a critical pain point in generative models for single-cell perturbation prediction. A good fit to the overall distribution does not guarantee accurate prediction of the perturbation effect. This insight aligns well with biological intuition and is statistically profound.
- This paper defines each particle in Sequential Monte Carlo (SMC) as a complete cell population rather than a single cell, which inherently matches the nature of single-cell sequencing as a "distributional comparison." This idea breaks the limitation of conventional test-time augmentation (TTA), which can only evaluate individual cells.
- The model requires no retraining, emphasizing a training-free (test-time alignment) approach offers substantial practical value.
Critical weaknesses
- The core assumptions are overly idealized. For instance, the perturbation effect is simplified to a uniform translation, which may not hold in complex biological systems where perturbation-induced changes in variance or cell-state composition are often present.
- The computational overhead caused by blindly increasing the number of particles at test time is substantial. This raises concerns about the model's cost-effectiveness in real-world large-scale applications.
Questions for authors
- The comparative experiments are insufficient. The baselines primarily focus on PerturbDiff (the original model), STATE, CellFlow, etc. Why was AdaCell not applied to recent large single-cell foundation models? The generalizability of the method remains unverified.
- The Anchor Reward in Equation (5) treats the perturbation-induced distributional change as a simple translation. However, in real biology, perturbations are often accompanied by changes in variance (heteroscedasticity) and even the depletion or emergence of cell subpopulations. Forcing alignment under a translation assumption may lead to over-regularization.
- Direct, gradient-free brute-force resampling is often highly inefficient and prone to getting trapped in local optima when the generative space is of extremely high dimensionality (e.g., 2,000 genes).
- As mentioned in Table 10 / Figure 5c, generating 1,000 cells using 16 particles takes 726.5 seconds. In the single-cell field, where generating hundreds of thousands of cells is routinely required, this time cost is prohibitive. The cost-effectiveness of the method warrants further consideration. It is necessary to test whether using only 4 (or even 2) particles can achieve better performance than the baseline with only a marginal increase in computation.
- Although Appendix D.2 presents a case study on the PAF1 gene, it merely demonstrates "more accurate prediction" without using AlphaFold or bioinformatics approaches to explain why the prediction is accurate.
5 comments
Dear Reviewer Ete8:
Thank you for these questions and suggestions.
Q1. Applicability to other backbones
We test Squidiff on 380 Replogle perturbations with matched controls, genes and evaluation. Squidiff is a different architecture, whereas Scratch and Finetuned are PerturbDiff training regimes. Scores are mean ± SD over seeds 42, 43 and 44.
| Squidiff inference | DEOver | PDCorr | PDS-cos | MSE |
|---|---|---|---|---|
| Native K=1 | 0.043±0.0024 | 0.081±0.0036 | 0.543±0.0051 | 4.255±0.0860 |
| AdaCell K=4 | 0.092±0.0019 | 0.106±0.0028 | 0.582±0.0037 | 4.018±0.0720 |
All metrics improve, but K=4 versus K=1 is not compute-matched and Squidiff is not a foundation-model test. Transfer requires conditional generation, clean gene-space estimates and resampleable particle states. We will clarify these requirements and limit claims to tested backbones.
Q2. Can the anchor over-regularize predictions
Our MMD anchor [1] is a soft preference, not a hard translation constraint, but may still bias distributional structure. We compare predictions with observed treated cells using matched cell counts and fixed projections. Sliced W1 averages Wasserstein-1 distances over one-dimensional projections. Centered W1 removes each population's mean before this comparison to assess differences beyond mean shifts. Gene-variance error measures discrepancies in per-gene variances. Variance ratio divides overall predicted variance by observed variance. Lower is better for the first three metrics, while the ratio targets one.
| Method | Sliced W1 | Centered W1 | Gene-variance error | Variance ratio |
|---|---|---|---|---|
| Native | 0.151 | 0.110 | 0.104 | 1.582 |
| Prior-only | 0.105 | 0.063 | 0.035 | 0.923 |
| AdaCell-16 | 0.133 | 0.097 | 0.100 | 1.741 |
AdaCell improves both W1 distances and gene-variance error over Native, but its higher variance ratio indicates greater overdispersion. Prior-only is closest to observed cells on all four diagnostics. These comparisons do not isolate the anchor's effect. We will report this trade-off without claiming accurate cell-state proportions.
Q3. Search in high-dimensional gene space
We use SMC-inspired reweighting and resampling [2] along model-proposed denoising trajectories. We compare intermediate steering with final-only selection at K=16. Replogle Scratch results are averaged over the same three seeds. AdaCell improves all four means over Best-of-16. Full AdaCell averages 1.670 resampling events and 7.552 distinct terminal ancestors, versus 1.402 and 8.204 without the anchor. With deterministic DDIM, copies can remain identical, so ancestry measures trajectory diversity, not cellular heterogeneity. These results support intermediate steering at fixed K, not global optimality or superiority to gradient-based search.
| Method | DEOver | PDCorr | PDS-cos | MSE |
|---|---|---|---|---|
| Best-of-16 | 0.184 | 0.363 | 0.599 | 0.012 |
| AdaCell-16 | 0.211 | 0.398 | 0.645 | 0.011 |
| Without anchor | 0.190 | 0.368 | 0.586 | 0.014 |
Q4. Performance with two or four particles
Scores are mean ± SD over seeds 42, 43 and 44. Timings are seconds per 1,000 valid cells using matched hardware, steps and precision, after warm-up and synchronization and excluding loading and evaluation. K=2 and K=4 improve DEOver and PDCorr but not PDS-cos. At K=2, ESS ≥ 1 prevents strict ESS < 0.5K resampling, so this setting uses terminal selection without intermediate resampling.
| Method | DEOver | PDCorr | PDS-cos | Time |
|---|---|---|---|---|
| Native K=1 | 0.174±0.0028 | 0.307±0.0037 | 0.630±0.0045 | 88.3 |
| AdaCell K=2 | 0.194±0.0035 | 0.363±0.0036 | 0.598±0.0007 | 118.3 |
| AdaCell K=4 | 0.196±0.0012 | 0.368±0.0028 | 0.606±0.0021 | 198.3 |
| AdaCell K=8 | 0.206±0.0010 | 0.382±0.0030 | 0.624±0.0082 | 376.6 |
| AdaCell K=16 | 0.211±0.0022 | 0.398±0.0011 | 0.645±0.0015 | 726.5 |
Q5. Biological interpretation of PAF1
PAF1 regulates Pol II promoter-proximal pausing [3] and enhancer-associated pause release [4]. These functions motivate assessing coordinated responses using training-derived expression priors. Relative to native PerturbDiff, DEOver rises from 0.212 to 0.496, PDCorr from 0.429 to 0.626, direction agreement from 0.714 to 0.861 and LFC correlation from 0.322 to 0.718, while MSE falls by 19.8%. These gains support better recovery of affected genes, response signs and effect ordering, but do not establish a pause-release mechanism in HepG2. Pathway analysis would more directly address the programs underlying this improvement, whereas AlphaFold alone cannot establish that link. We will state this limitation and that PAF1 was selected retrospectively for the largest DEOver gain.
[1] Gretton A et al. A Kernel Two-Sample Test. Journal of Machine Learning Research.
[2] Del Moral P, Doucet A, Jasra A. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society Series B (Statistical Methodology).
[3] Chen FX et al. PAF1, a Molecular Regulator of Promoter-Proximal Pausing by RNA Polymerase II. Cell.
[4] Chen FX et al. PAF1 regulation of promoter-proximal pause release via enhancer activation. Science.
Although the inference costs under different k values were tested, it is necessary to consider whether giving up accuracy for the sake of cost is reasonable? And how does it compare with the current technological level? The author is best served by having insights and analyses within the field.
Dear Reviewer Ete8:
We thank you for recognizing the motivation of our work and for your helpful comments on biological assumptions and computational cost. We clarify the role of the anchor and provide additional comparisons below.
W1. Uniform translation and biological variation
We use translation to construct a reference for the anchor reward, not to restrict the populations AdaCell can generate. Candidates come from the frozen conditional diffusion model, and we use maximum mean discrepancy (MMD) [1] to compare them with controls shifted by the response prior. This soft reference allows us to use observed control heterogeneity when the prior estimates only a mean response. The generated populations can still differ from controls in covariance and composition, and no reward assigns every cell a common expression profile.
We agree that a translated reference may be less appropriate when a perturbation substantially changes variance or cell-state proportions. To assess this directly, we compare full AdaCell with an otherwise identical no-anchor variant against observed treated cells in Q2. Including the anchor improves both Wasserstein measures but worsens the variance diagnostics, revealing a trade-off rather than uniform improvement in distributional fidelity. We will clarify that the anchor is a soft regularizer, report this trade-off explicitly, and avoid interpreting these measurements as evidence that cell-state proportions are accurately preserved.
W2. Computational overhead and practical scope
We agree that inference cost is important for practical use. Although AdaCell requires no backbone retraining, K=16 increases runtime from 88.3 to 726.5 seconds per 1,000 valid cells, an 8.23-fold cost relative to native PerturbDiff. Following your suggestion, we evaluate smaller budgets in Q4. K=2 and K=4 take 118.3 and 198.3 seconds, corresponding to 1.34 and 2.25 times native inference. Both improve DEOver and PDCorr, although PDS-cos remains below native. These results provide lower-cost options while making the metric-specific trade-off explicit.
To distinguish the benefit of steering from simply generating more candidates, we also compare AdaCell with Best-of-16 at the same candidate count in Q3. AdaCell improves all four mean response metrics, supporting the value of intermediate steering over terminal selection alone. We will present this matched-K comparison alongside the runtime analysis so that readers can assess both the benefit of steering and its additional cost. We will also distinguish candidate-count matching from wall-clock matching and make clear that atlas-scale practicality has not yet been demonstrated.