acceptodds
Under review as a conference paper at ICLR 2027

PerturbCellRL: Aligning Distributions and Grounding Biology via Post-Training Perturbation Generators

Abstract

Single-cell perturbation models can reduce costly wet-lab screening by predicting how cells respond transcriptionally to interventions. Recent advances in flow-matching have enabled population-level prediction of cellular responses. However, flow-matching training can fail to recover certain target distributions even within the model family, limiting its ability to capture cellular heterogeneity. We first prove that post-training can recover these distributions, then introduce PerturbCellRL, a reinforcement learning framework that post-trains single-cell perturbation generators using per-cell rewards. The central component is a gene-expression energy witness that translates population-level discrepancies into per-cell feedback. We further prove that this reward's policy gradient points toward better distributional alignment. Two complementary rewards, calibrated on real cells, penalize atypical expression profiles and insufficient pathway-level responses to perturbations. Across genetic and chemical perturbation benchmarks, PerturbCellRL substantially improves distributional alignment and recovers pathway enrichment patterns more faithfully. These results establish reward-guided post-training as an effective strategy for improving both distributional accuracy and biological fidelity in perturbation prediction.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Single-Cell Perturbation Modeling, Reinforcement Learning, Trustworthy AI

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LdNhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PerturbCellRL, an RL-based post-training framework for single-cell perturbation generation. It combines three rewards—Energy Witness, Expression Support, and Pathway Discrimination—to address suboptimal distribution fitting in pretrained Flow Matching models. Experiments demonstrate improved population-level distribution alignment and biological plausibility.

Critical strengths
  1. Complementary rewards. The three rewards encourage distributional coverage and reduce overconcentration, statistical expression plausibility, and biological consistency using pathway knowledge.

  2. Theoretical support. Energy Witness is theoretically linked to minimizing population-level distributional discrepancy.

  3. Experimental validation. Extensive evaluations and ablations on genetic and chemical perturbation datasets support the method's effectiveness.

Critical weaknesses

The motivation regarding cellular response heterogeneity in the Introduction could be misleading and would benefit from clarification.

The authors motivate their work by noting that the same mean expression change may result from weak responses across many cells or strong responses in a small subset. However, this example may lead readers to interpret the method as learning how individual cells respond differently based on their initial states.

During scDFM FM training, control cells uu and perturbed cells x1x_1 are independently sampled without cell-level correspondence. Consequently, for a given perturbation cc, any control cell uu can be paired with samples from the same target distribution PcP_c, providing no direct supervision for state-dependent transformations. Under this independent-sampling assumption, the theoretical optimum of the standard FM objective can therefore ignore uu entirely.

Moreover, none of the three rewards directly constrains the relationship between initial cell states and their responses. Therefore, reproducing heterogeneous response patterns does not necessarily imply that the model has learned which initial cells are more likely to respond strongly.

Although the paper acknowledges the unpaired nature of the data, the Introduction could more clearly distinguish heterogeneity in post-perturbation cell populations from initial-state-dependent individual response heterogeneity. Clarifying this distinction would better align the motivation with the method's actual scope.

Questions for authors

Could the authors clarify the motivation for retaining the initial control cell u as a conditioning input, given the unpaired nature of the training data?

In particular, does PerturbCellRL improve the model's ability to capture individual differences in u compared with the pretrained scDFM baseline? An analysis of how different control cells affect the generated responses under the same perturbation would help clarify whether the improvements extend beyond population-level distribution fitting.

3 comments
AuthorsOpenReview ↗

Response to Reviewer LdNh

We thank the reviewer for the encouraging feedback. We appreciate the reviewer's recognition of the complementary rewards, the theoretical link between the witness and distributional alignment, and the extensive evaluations. As suggested, we rewrote the heterogeneity example, stated the role of uu in Sec. 3.1, and analyzed uu-dependence empirically (Q3), which we have also added to the paper as the new App. I.

Q1. Population-level vs. initial-state-dependent heterogeneity.

We agree that the distinction should be explicit. Our prediction target is the distribution of the perturbed population PcP_c, not how each initial state uu responds. The example contrasts two perturbed populations with the same mean. These differ in distribution, so they can be distinguished from unpaired data: Energy and MMD separate them, whereas mean-based metrics cannot. Accordingly, our objective (Eq. 3) and all metrics are defined on the generated population Qθ,cQ_{\theta,c}. As the reviewer notes, neither the FM loss nor our rewards constrain the map from uu to its response, and none of our claims relies on it. The revised Introduction now reads: "…the same shift in average expression could come from a perturbed population in which all cells shift slightly, or one in which a small subset shifts strongly. […] the data reveal how the perturbed population is distributed, but not how each individual cell would have responded."

Q2. Why retain uu as an input?

We retain uu for two reasons:

  • Controlled comparison. Keeping uu lets PerturbCellRL post-train scDFM unchanged, so the gains in Table 1 come from post-training alone.
  • Multi-context datasets. uu carries the cellular context when a dataset spans several cell lines, cell types, or donors, so the same post-training applies unchanged there.

In scDFM, the flow starts from Gaussian noise and uu enters only as a conditioning input; it does not map each control cell to a perturbed counterpart. Since uu and x1x_1 are sampled independently given cc, the flow-matching optimum v⋆=E[x1−x0∣xt,t,c]v^{\star}=\mathbb{E}[x_1-x_0\mid x_t,t,c] does not depend on uu within a context, as the reviewer notes. The revised Sec. 3.1 now states this role of uu.

Q3. Does post-training change dependence on individual uu?

No. Neither scDFM nor PerturbCellRL depends detectably on individual differences in uu, as expected from the flow-matching optimum.

We test whether each generated cell correlates more with its own control cell uu than with other control cells, as it would if the output depended on uu. For each held-out condition, each model generates 128 cells, one from each of 128 control cells, with the same control cells and noise for both models. After subtracting the condition mean, we average the correlation between each generated cell and its own control cell, and compare it with random pairings (permutation test, 199 permutations). We ran this on all 308 held-out conditions, counting each Norman fold separately. The test is sensitive: when we add an artificial uu-dependence to scDFM's outputs, it detects it in every condition once that dependence explains about 0.5% of the variation among generated cells. Since this correlation captures only dependence along the same genes, we also run a distance-correlation test between the control cells and the generated cells (top 10 principal components of each, 199 permutations), which also captures nonlinear and cross-gene dependence.

ModelCorrelation with own uuCorrelation test: conditions with p<0.05p<0.05Distance-correlation test: conditions with p<0.05p<0.05
scDFM0.0016.2%2.9%
PerturbCellRL0.0026.2%3.2%

Both models are at chance level in both tests: about 5% of conditions would reach p<0.05p<0.05 if the output ignored uu. The variation among generated cells therefore comes from the noise draw, and the gains of PerturbCellRL are at the population level. Learning which control states respond more strongly would require paired or lineage-resolved measurements, which these screens do not provide; we see this as an interesting direction for future work.

We hope these clarifications address the reviewer's concerns and would be glad to answer any further questions.

Reviewer_LdNhOpenReview ↗

Response

I appreciate the authors' response, which addresses most of my concerns. However, the new experiments confirm no detectable dependence on individual uu. While retaining uu ensures consistency with scDFM, this does not establish its necessity. Its potential benefits in multi-context settings also remain unverified. I believe this limitation should be explicitly acknowledged, as retaining an unvalidated conditioning input warrants stronger justification.

AuthorsOpenReview ↗

Response to Reviewer LdNh (Part 2)

We thank the reviewer for the follow-up and agree that this should be stated explicitly. PerturbCellRL's contribution lies in the post-training process, not the base model. We therefore keep scDFM's inputs, including uu, unchanged, so that the gains come from post-training alone. We do not claim that uu is necessary, nor that post-trained scDFM is the best possible formulation, and whether uu helps in multi-context settings remains untested.

PerturbCellRL also applies to other single-cell flow-matching models. In the revised PDF, we added a new experiment on a second base model, scLDM (App. G), which has no control-cell input. Post-training improves all nine metrics there as well, so the gains generalize beyond scDFM and do not rely on uu. Mean ±\pm std over 3 generation seeds on scLDM's Replogle benchmark:

ModelEnergy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrow
scLDM2.3945 ±\pm 0.01590.0154 ±\pm 0.00030.1979 ±\pm 0.00070.3706 ±\pm 0.00600.1329 ±\pm 0.00060.3978 ±\pm 0.00260.3167 ±\pm 0.00560.3270 ±\pm 0.00160.7518 ±\pm 0.0032
+ PerturbCellRL2.1088 ±\pm 0.01210.0098 ±\pm 0.00020.1829 ±\pm 0.00050.4030 ±\pm 0.00350.1210 ±\pm 0.00040.4304 ±\pm 0.00200.3369 ±\pm 0.00450.3514 ±\pm 0.00070.7617 ±\pm 0.0044

As suggested, we now state this in the Limitations: "We retain scDFM's control-cell input uu to post-train it unchanged. Within a context, the flow-matching optimum does not depend on uu, and whether uu helps in multi-context settings remains untested. Our second base model, scLDM, has no such input (Appendix G)." Sec. 3.1 now likewise describes uu only as an input inherited from scDFM.

Reviewer_U71yOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

PerturbCellRL proposes a reward-guided post-training framework for single-cell perturbation generators. It combines an energy-witness reward for population-level distribution alignment with expression-support and pathway-discrimination rewards, using scDFM and DiffusionNFT as the starting point. Experiments on Norman and ComboSciPlex show consistent improvements in distributional metrics, with smaller gains in pathway-related metrics.

Critical strengths
  1. The paper addresses an important problem: matching the distribution of generated cells rather than only their average expression. The energy-witness reward provides a principled way to encourage generated populations to better match observed cells.

  2. The method improves Energy distance, MMD, and sliced Wasserstein distance over the pretrained scDFM across the reported settings. The paper also provides useful implementation details and ablations to examine the roles of different rewards.

Critical weaknesses
  1. The advantage of RL post-training over direct distribution optimization is not established. The witness-only variant achieves better Energy distance than the full model in the reported Norman settings, but it still uses the same NFT framework. Since Energy distance can be optimized directly through generated endpoints, a matched non-RL fine-tuning baseline is needed to establish the value of the proposed optimization approach. The experiments also use only scDFM, leaving generality across generators unclear.

  2. The independent contribution of the biological rewards remains uncertain. Adding the support and pathway rewards does not consistently improve distributional or pathway metrics over witness-only. Moreover, the biological evaluations largely follow the same scoring criteria used to construct the rewards. Without independent metrics and complete reward ablations, it is difficult to determine whether these rewards improve biological fidelity beyond the witness reward.

  3. Distributional gains may come at the expense of perturbation specificity, and their stability is unclear. DS decreases across all three Norman settings, while DE-Sp. also declines. Some reported improvements in NES-Sp. are small, and results are presented without repeated-seed uncertainty estimates. These findings warrant a more careful assessment of the trade-off between population-level alignment and perturbation-specific prediction.

  4. The theoretical analysis does not fully justify the implemented algorithm. The main gradient identity is established under idealized assumptions, whereas the actual method uses finite-sample rewards, within-group standardization, clipping, auxiliary rewards, and an NFT surrogate objective. The paper does not show that these operations preserve the stated identity or guarantee a decrease in Energy distance. The theoretical results therefore motivate the reward design but do not yet provide a guarantee for the implemented method.

Questions for authors

1.Could the authors compare NFT post-training with direct differentiable Energy-distance fine-tuning under comparable optimization budgets and report computational costs?

  1. Could the authors provide complete reward ablations using independent metrics, including NES-Sp., DE-Sp., and DS, to isolate the contributions of the witness, support, and pathway rewards?

  2. Could the authors report results from multiple training seeds with uncertainty estimates, and analyze the consistent decreases in DS and DE-Sp.?

  3. How do the idealized gradient results relate to the implemented NFT updates with reward standardization and clipping? Can the authors verify whether the actual updates reduce Energy distance in practice?

8 comments
AuthorsOpenReview ↗

Response to Reviewer U71y (Part 1)

We thank the reviewer for the constructive feedback. We appreciate that the reviewer considers distribution matching an important problem, finds the witness reward principled, and notes the consistent distributional gains. We ran each requested experiment and added a proof relating the gradient identity to the implemented NFT update (new App. H). We have also added the direct Energy fine-tuning comparison (Q1), the training-seed analysis (Q4), and the Energy trend during post-training (Q6) to the paper as the new App. I.

Q1. NFT vs. direct differentiable Energy fine-tuning.

As suggested, we replaced NFT with fine-tuning that backpropagates the energy distance through the RK4 sampler (20 time points, 19 steps), keeping everything else identical to the witness-only run on ComboSciPlex. At a comparable compute budget, NFT is better than or tied with direct fine-tuning on all nine metrics:

  • Cost. Direct fine-tuning runs out of memory on an 80 GB H100 without gradient checkpointing, and each step costs 5.6$\times$ an NFT step.
  • Applicability. The support and pathway rewards are calibrated tail ranks and are not differentiable. Direct fine-tuning can therefore optimize only the witness term, whereas NFT optimizes the full reward of Eq. (15).
MethodSteps / GPU timeEnergy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrow
scDFM–0.51770.04620.06540.89190.02420.83630.84180.86790.8776
Direct Energy fine-tuning40 / 48 min0.45290.03610.05860.89050.02340.84030.84840.87250.8980
NFT, witness only200 / 43 min0.44130.03580.05810.90280.02310.84100.85670.87970.8980

Q2. Generality across generators.

We post-trained a second base model, scLDM [1] (new App. G). scLDM differs from scDFM in its main design choices: flow matching in a VAE latent space, a condition token instead of a control cell, and a negative binomial decoder. We kept its released checkpoints, splits, sampler, and default guidance weight, and changed only the post-training step. On scLDM's Replogle benchmark, each of the 372 test perturbations is evaluated in HepG2 but seen in training only in other cell lines. Post-training with all three rewards improves all 9 metrics. Mean ±\pm std over 3 generation seeds:

ModelEnergy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrow
scLDM2.3945 ±\pm 0.01590.0154 ±\pm 0.00030.1979 ±\pm 0.00070.3706 ±\pm 0.00600.1329 ±\pm 0.00060.3978 ±\pm 0.00260.3167 ±\pm 0.00560.3270 ±\pm 0.00160.7518 ±\pm 0.0032
+ PerturbCellRL2.1088 ±\pm 0.01210.0098 ±\pm 0.00020.1829 ±\pm 0.00050.4030 ±\pm 0.00350.1210 ±\pm 0.00040.4304 ±\pm 0.00200.3369 ±\pm 0.00450.3514 ±\pm 0.00070.7617 ±\pm 0.0044
AuthorsOpenReview ↗

Response to Reviewer U71y (Part 2)

Q3. Independent contribution of the biological rewards.

We re-ran the single-reward ablation on all four settings and all nine Table 1 metrics; it is now Table 2 in the main text. Each reward primarily improves a different category:

  • Witness → distribution. Alone, it gives the lowest MMD and SW1 in all four settings and the lowest Energy in three.
  • Pathway → pathway level and perturbation specificity. Alone, it improves NES-Sp., DE-Sp., and ρ(Δ)\rho(\Delta) over scDFM in all four settings, and gives the highest DS and per-cell pathway score on all three Norman settings.
  • Support → single-cell level. Alone, it gives the highest per-cell support score on all three Norman settings.
  • Combined. The full reward is best or second best on all distribution metrics in every setting, and it recovers part of the DS that the witness alone loses on Norman holdout.

On independence: none of the nine Table 1 metrics is a training reward, and all are computed on held-out perturbations. NES-Sp. uses Reactome GSEA, which is independent of the Hallmark-based pathway reward. The biological rewards are not designed to improve distributional metrics over the witness; they target per-cell plausibility and perturbation specificity. On scLDM (App. G, Table 9), only the combined reward improves all nine metrics over the base model; the witness alone, for example, lowers DS from 0.752 to 0.695.

In the table, sup and path are the per-cell support and pathway scores, which are the training rewards. Norman settings are four-fold means. Within each setting, bold is best and underline second best.

RewardEnergy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrowsup ↑\uparrowpath ↑\uparrow
Additive
scDFM0.55620.04590.07380.78760.02320.90170.83330.88200.9743‾\underline{0.9743}0.90880.9464
witness only0.47940.03580.06650.80030.0220‾\underline{0.0220}0.90380.84950.89080.95670.92830.9474‾\underline{0.9474}
support only0.51740.04090.07010.78640.02240.89880.84100.88480.96060.94110.9451
pathway only0.56010.04590.07380.79160.02340.9026‾\underline{0.9026}0.83330.88360.97750.90770.9504
PerturbCellRL0.4865‾\underline{0.4865}0.0367‾\underline{0.0367}0.0672‾\underline{0.0672}0.7973‾\underline{0.7973}0.02190.90070.8479‾\underline{0.8479}0.8895‾\underline{0.8895}0.95580.9395‾\underline{0.9395}0.9470
Holdout single
scDFM0.56420.05810.07360.59470.02030.8372‾\underline{0.8372}0.68480.71820.8904‾\underline{0.8904}0.92640.9079
witness only0.49650.04670.06640.61300.01940.83190.71180.72390.85320.94740.9102
support only0.53550.05250.07020.58510.01990.83000.69920.71700.86000.95960.9072
pathway only0.56720.05850.07390.6136‾\underline{0.6136}0.02040.84170.67860.72900.89490.92310.9156
PerturbCellRL0.5021‾\underline{0.5021}0.0478‾\underline{0.0478}0.0671‾\underline{0.0671}0.61530.0195‾\underline{0.0195}0.83420.7099‾\underline{0.7099}0.7281‾\underline{0.7281}0.86300.9503‾\underline{0.9503}0.9128‾\underline{0.9128}
Holdout double
scDFM0.63980.04550.07670.76020.02510.8845‾\underline{0.8845}0.74350.82790.9144‾\underline{0.9144}0.89780.9250
witness only0.5979‾\underline{0.5979}0.03850.07160.76330.0247‾\underline{0.0247}0.87720.74930.82900.88220.92150.9264
support only0.62510.04200.07430.74710.02500.87450.74320.82540.88110.93770.9238
pathway only0.63840.04560.07680.77550.02510.88770.7498‾\underline{0.7498}0.83370.93780.89160.9328
PerturbCellRL0.59610.0388‾\underline{0.0388}0.0718‾\underline{0.0718}0.7657‾\underline{0.7657}0.02460.87770.75260.8312‾\underline{0.8312}0.89110.9260‾\underline{0.9260}0.9295‾\underline{0.9295}
ComboSciPlex
scDFM0.51770.04620.06540.89190.02420.83630.84180.86790.8776‾\underline{0.8776}0.93050.8735
witness only0.44130.03580.05810.90280.02310.8410‾\underline{0.8410}0.85670.87970.89800.95360.8811
support only0.48520.04100.06200.89350.0237‾\underline{0.0237}0.83980.84420.86890.8776‾\underline{0.8776}0.9600‾\underline{0.9600}0.8775
pathway only0.50130.04420.06410.8991‾\underline{0.8991}0.02400.83950.84770.87370.8776‾\underline{0.8776}0.93490.8791
PerturbCellRL0.4498‾\underline{0.4498}0.0366‾\underline{0.0366}0.0587‾\underline{0.0587}0.89860.02310.84160.8536‾\underline{0.8536}0.8770‾\underline{0.8770}0.89800.96320.8809‾\underline{0.8809}
AuthorsOpenReview ↗

Response to Reviewer U71y (Part 3)

Q4. Multiple training seeds.

We retrained PerturbCellRL on ComboSciPlex with three training seeds. For every metric, the mean gain over scDFM is at least 3.8× the standard deviation across seeds, and at least 10× for all metrics other than NES-Sp. For NES-Sp., each of the three seeds (0.8968, 0.9003, 0.8992) still exceeds scDFM’s 0.8919. DS is identical across seeds because ComboSciPlex has only 7 test conditions. Mean ±\pm std over three training seeds:

ModelEnergy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrow
scDFM0.51770.04620.06540.89190.02420.83630.84180.86790.8776
PerturbCellRL0.4470 ±\pm 0.00260.0362 ±\pm 0.00030.0585 ±\pm 0.00020.8988 ±\pm 0.00180.0231 ±\pm 0.00010.8423 ±\pm 0.00060.8542 ±\pm 0.00080.8774 ±\pm 0.00060.8980 ±\pm 0.0000

On Norman, each of the four folds is a separate post-training run with its own test set. We therefore report the paired gain Δ\Delta (PerturbCellRL −- scDFM within each fold) as mean ±\pm std over folds:

Δ\Delta (paired)Energy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrow
Additive$-$0.0697 ±\pm 0.0170$-$0.0092 ±\pm 0.0013$-$0.0066 ±\pm 0.0011+0.0097 ±\pm 0.0078$-$0.0013 ±\pm 0.0005$-$0.0011 ±\pm 0.0016+0.0146 ±\pm 0.0065+0.0074 ±\pm 0.0045$-$0.0184 ±\pm 0.0133
Holdout single$-$0.0622 ±\pm 0.0142$-$0.0103 ±\pm 0.0020$-$0.0066 ±\pm 0.0012+0.0206 ±\pm 0.0092$-$0.0009 ±\pm 0.0005$-$0.0030 ±\pm 0.0033+0.0251 ±\pm 0.0079+0.0099 ±\pm 0.0058$-$0.0274 ±\pm 0.0256
Holdout double$-$0.0436 ±\pm 0.0256$-$0.0067 ±\pm 0.0030$-$0.0049 ±\pm 0.0017+0.0055 ±\pm 0.0107$-$0.0005 ±\pm 0.0003$-$0.0067 ±\pm 0.0036+0.0092 ±\pm 0.0122+0.0033 ±\pm 0.0076$-$0.0233 ±\pm 0.0222

Energy, MMD, SW1, and MAE improve in every fold of all three Norman settings. NES-Sp. improves in every fold of additive and holdout single. On holdout double, it improves in two folds (+0.017, +0.012) and is slightly lower in two ($-$0.003, $-$0.004).

Q5. Trade-off with perturbation specificity (DS, DE-Sp.).

The trade-off is small and confined to DS and DE-Sp. on Norman, where the 6.8–12.5% improvement in Energy comes with a 1.9–3.1% decrease in DS and a 0.1–0.8% decrease in DE-Sp. All other metrics improve in all four settings, and on ComboSciPlex DS and DE-Sp. improve as well.

  • Source and control. The ablation (Q3) traces the drop to the witness reward, while the pathway reward alone improves both DS and DE-Sp. over scDFM on all three Norman settings. This suggests the balance can be shifted toward perturbation specificity by increasing λpath\lambda_{\mathrm{path}}; our reported weights were selected on the validation set for overall performance (App. C).
  • Why distributional and single-cell fidelity matter. Perturbation responses are heterogeneous [2, 3]. The same mean shift can come from all cells shifting slightly or from a subset shifting strongly. DE-Sp. and DS, computed on pseudobulk means, score these two populations identically, whereas Energy and MMD distinguish them. A recent benchmark recommends evaluating distributional calibration alongside mean accuracy [4]. At the single-cell level, the energy distance scales with the square of the fraction of implausible cells, so 5% wrong cells change it by only 0.25% of a fully wrong population. Yet such cells can form spurious clusters or bias per-cell calls such as Mixscape [5]; the support and pathway rewards target them. The revised Introduction now states the distributional motivation.
Reviewer_GyMsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an RL post-training approach for a pretrained flow-matching model (scDFM), leveraging the previously proposed DiffusionNFT. The paper's contributions are single-cell reward functions based on the energy witness, expression support, and pathway discrimination. The theoretical analysis shows that the policy gradient of the energy-witness reward equals, up to a constant factor, the gradient of the population-level energy distance, and illustrates on a Gaussian example how flow matching can miss the target distribution. Experimental results on two single-context datasets, one genetic and one chemical, show improved distributional performance over scDFM and mixed results on perturbation-effect metrics (DE-Spearman, DS, Pearson delta).

Critical strengths
  • The paper's focus on complementing the flow-matching objective with reward-based post-training tailored to single-cell data is an interesting direction.
  • The proposed approach improves over scDFM on distributional metrics.
  • The theoretical analysis may be of interest to researchers working on post-training of flow-matching models.
Critical weaknesses
  • Limited novelty: The main contribution appears to be relatively straightforward reward functions applied to a previously proposed pretrained model (scDFM), optimized with an existing RL method (DiffusionNFT). Moreover, since Theorem 1 shows that the witness reward's policy gradient equals (up to a factor) the energy-distance gradient, it is unclear why both terms are included in Eq. (3), and whether Theorem 1 applies to the DiffusionNFT update used in the paper.
  • Overstated theoretical contributions:
    • Proposition 1: The mismatch between the flow-matching solution and the target distribution appears to stem from a misspecified velocity family. The paper should state this condition explicitly and clarify when such mismatch arises in practice.
    • Theorem 1: The result is a straightforward combination of the score-function (REINFORCE) identity and energy-distance gradients, and the energy witness being the first variation of the energy distance is well known [2]. It would be more accurately presented as a lemma or proposition.
  • Gains versus added complexity: The improvements are mainly on distributional metrics, while results on perturbation-effect metrics (DE-Spearman, DS, Pearson delta), which are standard for assessing perturbation-specific effects, are mixed. A complexity analysis is needed to justify the additional post-training cost.
  • Limited evaluation:
    • The approach appears applicable to other flow-matching single-cell models, such as scLDM [1], yet it is evaluated only on scDFM. Applying it to an additional model would demonstrate its generality. It would also be informative to decompose the gains attributable to the base model, the rewards, and the RL procedure.
    • The reward functions appear to overlap in purpose, yet the ablation study is limited to the Norman dataset and to metrics aligned with each reward's objective. Since the rewards are a main contribution, the ablation should cover all datasets and metrics in Table 1. Moreover, the motivation and implementation of the rewards beyond the energy witness are difficult to follow.
    • It is unclear whether classifier-free guidance would achieve similar results. Given its simplicity, the paper should include a tuned guidance baseline (e.g., as used in scLDM [1]).
    • The evaluation is limited to two single-context datasets inherited from scDFM. Evaluating whether the gains transfer to cross-context settings (e.g., unseen cell types or donors) would strengthen the submission. In addition, the paper does not specify how the ComboSciPlex splits are defined.

References

  • [1] Palla et al., Scalable single-cell gene expression generation with latent diffusion models. In ICML, 2026.
  • [2] Bellemare et al., The Cramér distance as a solution to biased Wasserstein gradients. arXiv, 2017.
Questions for authors

See weaknesses

10 comments
AuthorsOpenReview ↗

Response to Reviewer GyMs (Part 1)

We thank the reviewer for the constructive feedback. We appreciate that the reviewer finds reward-based post-training for single-cell data an interesting direction and values the distributional gains over scDFM and the theoretical analysis. In response, we added a second base model (scLDM), a complete reward ablation, a tuned guidance baseline (new App. I), a cost analysis, and a proof that energy-distance descent carries over to a functional NFT step (new App. H).

Q1. Novelty beyond scDFM + DiffusionNFT.

PerturbCellRL deliberately builds on existing frameworks. Our contribution is addressing what breaks when RL post-training is transferred naively to virtual cell modeling. RL scores each generated cell separately, but downstream use depends on two things: matching the heterogeneous perturbed population, and avoiding individual cells far outside those measured under the same perturbation. We therefore design per-cell rewards that serve both levels:

  • Witness (population level). Rewarding each cell only for closeness to real cells can collapse the population and erase heterogeneity. The energy witness adds a repulsion among generated cells, so its per-cell policy gradient descends the population energy distance (Theorem 1, now Prop. 2). This carries over to the NFT update (Q3).
  • Support and pathway (single-cell level). The energy distance barely responds to a few implausible cells (Q2). These rewards therefore penalize only cells that score worse than typical held-out real cells, in expression profile or pathway response. Each reward primarily improves a different category of metrics (Q8).
  • Efficiency and generality. Post-training takes 0.75–1.5 H100-hours with no inference cost (Q6), and it improves all 9 metrics on a second base model, scLDM (Q7). Together, this makes PerturbCellRL an efficient and generalizable post-training framework for flow-matching virtual cell models.

Q2. Why both terms in Eq. (3), given Theorem 1?

The two terms play different roles. The witness reward is how we optimize the population term DD, while rbior_{\rm bio} scores each generated cell against held-out real cells. DD alone is insensitive to a small fraction of implausible cells: if Q=(1−ε)P+εRQ=(1-\varepsilon)P+\varepsilon R, then D(Q,P)=ε2D(R,P)D(Q,P)=\varepsilon^2 D(R,P). With ε=5\varepsilon=5%, such cells contribute only 0.25% of D(R,P)D(R,P). Yet they can form a spurious cluster that would be read as a perturbation-induced state, the same artifact that motivates doublet removal [1]. Since the KL constraint and finite capacity keep DD above zero in practice, this matters.

Consistent with this, the support and pathway rewards alone give the highest per-cell support and pathway scores on all Norman settings (Q8). The revised Sec. 3.1 now states the roles of the two terms right after Eq. (3), and the opening of Sec. 3.3.2 gives this ε2\varepsilon^2 argument.

Q3. Does Theorem 1 apply to the DiffusionNFT update?

Yes. A functional NFT step still decreases the energy distance at first order, at rate 2κ2\kappa (new App. H, Prop. 7). Each implementation choice enters through this single quantity κ\kappa, the within-group covariance between a cell's witness and its advantage:

  • NFT surrogate. For the NFT loss, the advantage-weighted flow-matching residual takes the place of the likelihood score in Prop. 2, and the same pairing again yields κ\kappa for any generator (Prop. 4). A functional NFT step of size ε\varepsilon therefore changes the energy distance by −2εκ-2\varepsilon\kappa at first order, up to an error term quantified in App. H (Prop. 7).
  • Implementation choices. With the exact witness, group standardization and our clipping keep κ>0\kappa>0 whenever the generated and real populations differ (Cor. 2). The finite-sample witness is unbiased and preserves this sign in expectation under symmetric estimation noise.

Q4. Condition behind Proposition 1.

We agree, and the revised Prop. 1 now states the condition explicitly.

The mismatch arises when the velocity family cannot represent the marginal velocity that flow matching regresses onto, even though the family of generated distributions contains the target. In App. B's Gaussian example, vb(x,t)=−bxv_b(x,t)=-bx generates the target exactly at b=1b=1, but no vbv_b equals the marginal velocity. Flow matching therefore selects b≈1.14b\approx1.14, which yields only 75.2% of the target variance.

In practice, a neural network generally cannot represent the marginal velocity exactly. Consistent with this, post-training lowers scDFM's Energy by 6.8–13.1% without changing the architecture (Table 1).

AuthorsOpenReview ↗

Response to Reviewer GyMs (Part 2)

Q5. Theorem 1 → proposition.

We agree and have renamed it Proposition 2. The proof in App. A of the original submission already followed and cited Bellemare et al. [2, App. A, Eq. 8], and the revised Sec. 3.3.1 now also states, next to the result, that it follows from the score-function identity and the first variation of the energy distance [2]. Our contribution is using this identity as a per-cell RL reward and extending it to the NFT update (Q3).

Q6. Complexity and cost.

Post-training is very efficient: it takes only 0.75–1.5 GPU-hours on a single H100 (200 iterations) and adds no inference cost, since the architecture and sampler are unchanged (App. C, new Table 4). Each iteration generates NN cells with an RK4 sampler on 20 time points (19 steps) and takes one NFT step, for O(NT)O(NT) network evaluations. The rollouts need 76 gradient-free passes per cell; the update needs one forward–backward pass per cell at a single time tt. There is no backpropagation through the sampler, and the rewards need no network evaluations. Measured on one H100, excluding periodic evaluation:

SettingCells / iterationTime / iterationTotal (200 iterations)
Norman additive6414.2 s47 min
Norman holdout128≈\approx 26 s≈\approx 1.5 h
ComboSciPlex6413.4 s45 min

Q7. Other base models (e.g., scLDM).

As suggested, we post-trained scLDM [3] (new App. G). scLDM differs from scDFM in its main design choices: flow matching in a VAE latent space, a condition token instead of a control cell, and a negative binomial decoder. We kept its released checkpoints, splits, sampler, and default guidance weight, and changed only the post-training step. On scLDM's Replogle benchmark, each of the 372 test perturbations is evaluated in HepG2 but seen in training only in other cell lines. Post-training with all three rewards improves all 9 metrics. Mean ±\pm std over 3 generation seeds:

ModelEnergy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrow
scLDM2.3945 ±\pm 0.01590.0154 ±\pm 0.00030.1979 ±\pm 0.00070.3706 ±\pm 0.00600.1329 ±\pm 0.00060.3978 ±\pm 0.00260.3167 ±\pm 0.00560.3270 ±\pm 0.00160.7518 ±\pm 0.0032
+ PerturbCellRL2.1088 ±\pm 0.01210.0098 ±\pm 0.00020.1829 ±\pm 0.00050.4030 ±\pm 0.00350.1210 ±\pm 0.00040.4304 ±\pm 0.00200.3369 ±\pm 0.00450.3514 ±\pm 0.00070.7617 ±\pm 0.0044
AuthorsOpenReview ↗

Response to Reviewer GyMs (Part 3)

Q8. Ablation on all datasets and metrics.

We re-ran the single-reward ablation on all four settings and all nine Table 1 metrics. It is now Table 2 in the main text. Each reward primarily improves a different category of metrics:

  • Witness → distribution. Alone, it gives the lowest MMD and SW1 in all four settings and the lowest Energy in three. It also improves NES-Sp., MAE, ρ(Δ^)\rho(\hat\Delta), and ρ(Δ)\rho(\Delta) over scDFM in all four.
  • Pathway → pathway level and perturbation specificity. Alone, it improves NES-Sp., DE-Sp., and ρ(Δ)\rho(\Delta) over scDFM in all four settings, and gives the highest DS and per-cell pathway score on all three Norman settings.
  • Support → single-cell level. Alone, it gives the highest per-cell support score on all three Norman settings. It also lowers Energy, MMD, SW1, and MAE relative to scDFM in all four.
  • Combined. The full reward is best or second best on all distribution metrics in every setting and keeps the witness's gains on NES-Sp., MAE, ρ(Δ^)\rho(\hat\Delta), and ρ(Δ)\rho(\Delta). It also recovers part of the DS that the witness alone loses on Norman holdout.

On scLDM (App. G, Table 9), the same pattern holds, and only the combined reward improves all nine metrics over the base model.

In the table, sup and path are the per-cell support and pathway scores, which are the training rewards. Norman settings are four-fold means. Within each setting, bold is best and underline second best.

RewardEnergy ↓\downarrowMMD ↓\downarrowSW1 ↓\downarrowNES-Sp. ↑\uparrowMAE ↓\downarrowDE-Sp. ↑\uparrowρ(Δ^)\rho(\hat\Delta) ↑\uparrowρ(Δ)\rho(\Delta) ↑\uparrowDS ↑\uparrowsup ↑\uparrowpath ↑\uparrow
Additive
scDFM0.55620.04590.07380.78760.02320.90170.83330.88200.9743‾\underline{0.9743}0.90880.9464
witness only0.47940.03580.06650.80030.0220‾\underline{0.0220}0.90380.84950.89080.95670.92830.9474‾\underline{0.9474}
support only0.51740.04090.07010.78640.02240.89880.84100.88480.96060.94110.9451
pathway only0.56010.04590.07380.79160.02340.9026‾\underline{0.9026}0.83330.88360.97750.90770.9504
PerturbCellRL0.4865‾\underline{0.4865}0.0367‾\underline{0.0367}0.0672‾\underline{0.0672}0.7973‾\underline{0.7973}0.02190.90070.8479‾\underline{0.8479}0.8895‾\underline{0.8895}0.95580.9395‾\underline{0.9395}0.9470
Holdout single
scDFM0.56420.05810.07360.59470.02030.8372‾\underline{0.8372}0.68480.71820.8904‾\underline{0.8904}0.92640.9079
witness only0.49650.04670.06640.61300.01940.83190.71180.72390.85320.94740.9102
support only0.53550.05250.07020.58510.01990.83000.69920.71700.86000.95960.9072
pathway only0.56720.05850.07390.6136‾\underline{0.6136}0.02040.84170.67860.72900.89490.92310.9156
PerturbCellRL0.5021‾\underline{0.5021}0.0478‾\underline{0.0478}0.0671‾\underline{0.0671}0.61530.0195‾\underline{0.0195}0.83420.7099‾\underline{0.7099}0.7281‾\underline{0.7281}0.86300.9503‾\underline{0.9503}0.9128‾\underline{0.9128}
Holdout double
scDFM0.63980.04550.07670.76020.02510.8845‾\underline{0.8845}0.74350.82790.9144‾\underline{0.9144}0.89780.9250
witness only0.5979‾\underline{0.5979}0.03850.07160.76330.0247‾\underline{0.0247}0.87720.74930.82900.88220.92150.9264
support only0.62510.04200.07430.74710.02500.87450.74320.82540.88110.93770.9238
pathway only0.63840.04560.07680.77550.02510.88770.7498‾\underline{0.7498}0.83370.93780.89160.9328
PerturbCellRL0.59610.0388‾\underline{0.0388}0.0718‾\underline{0.0718}0.7657‾\underline{0.7657}0.02460.87770.75260.8312‾\underline{0.8312}0.89110.9260‾\underline{0.9260}0.9295‾\underline{0.9295}
ComboSciPlex
scDFM0.51770.04620.06540.89190.02420.83630.84180.86790.8776‾\underline{0.8776}0.93050.8735
witness only0.44130.03580.05810.90280.02310.8410‾\underline{0.8410}0.85670.87970.89800.95360.8811
support only0.48520.04100.06200.89350.0237‾\underline{0.0237}0.83980.84420.86890.8776‾\underline{0.8776}0.9600‾\underline{0.9600}0.8775
pathway only0.50130.04420.06410.8991‾\underline{0.8991}0.02400.83950.84770.87370.8776‾\underline{0.8776}0.93490.8791
PerturbCellRL0.4498‾\underline{0.4498}0.0366‾\underline{0.0366}0.0587‾\underline{0.0587}0.89860.02310.84160.8536‾\underline{0.8536}0.8770‾\underline{0.8770}0.89800.96320.8809‾\underline{0.8809}

Q9. Decomposing base model, rewards, and RL procedure.

For the base model, see Q7; for the rewards, see Q8. On the RL procedure: the support and pathway rewards are rank-based and non-differentiable, so gradient-based fine-tuning cannot optimize them, whereas RL needs only reward values, which lets a single update optimize all three rewards.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.