- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Maximum Mean Discrepancy Policy Optimization (MMD-PO) for reward-guided post-training of virtual-cell models. The method reweights cell populations sampled from a frozen policy using biological rewards and a reference-distance penalty, then fits the current policy to the resulting empirical target through MMD. It combines gene-effect and pathway-enrichment rewards to improve perturbation prioritization. Evaluation includes retrospective experiments across three backbones, held-out HepG2 perturbation responses, transfer between resting and stimulated T cells, and controlled synthetic studies examining when post-training helps.
Critical strengths
-
A useful post-training interface for population-generating models. MMD-PO avoids evaluating output likelihoods and differentiating through the reward, while operating directly on generated cell populations. This is a practically relevant construction, and evaluation across multiple backbones provides some evidence of portability.
-
Promising prioritization results accompanied by boundary-condition analysis. Some improvements are substantial: for X-Cell on HepG2, EF@1% increases from 6.714 with GRPO to 10.300 with MMD-PO. Reporting both prioritization and predictive-fidelity metrics, together with controlled synthetic experiments, helps characterize the benefits and limitations of the approach.
Critical weaknesses
-
The missing direct task-loss baseline limits attribution of the central contribution. Both rewards measure agreement between predictions and observed responses. An important alternative is therefore to augment the native supervised objective with the same gene-effect and pathway-effect errors. The existing pathway-gene SFT baseline restricts the genes covered by the native loss, while reward-weighted SFT changes example weights; neither directly tests this alternative. At minimum, the differentiable Gene-Effect Huber loss could be optimized directly. For the pathway component, a clearly specified differentiable implementation or surrogate could be evaluated with validation-based loss-weight tuning. The same-reward GRPO/Flow-GRPO comparisons already provide partial attribution evidence, but they do not establish whether MMD-PO offers an advantage over this straightforward task-aligned optimization route. This distinction materially affects the claimed algorithmic contribution.
-
The additional benefit of the MMD update is not sufficiently isolated from the broader training recipe. The method combines candidate generation, reward weighting, reference-distance penalization, and MMD fitting. A comparison that holds the candidate populations and weights fixed while varying the fitting objective would help determine whether the benefit specifically comes from MMD or more generally from learning from reward-weighted generations. Moreover, the PerturbCellRL-inspired comparison omits an original reward component, and some compatibility experiments differ in rewards, optimization settings, or computational conditions. These limitations are disclosed, but the resulting comparisons do not cleanly establish superiority of the update mechanism over the closest likelihood-free alternatives. Stronger attribution is important for assessing the significance of the proposed algorithm.
Questions for authors
-
Can you hold the reward-weighted candidate populations fixed and compare MMD with alternative distribution-matching or weighted-fitting objectives? A comparison against a close likelihood-free baseline using the same reward, initialization, and a comparable training budget would also clarify the update rule’s independent contribution.
-
The practical algorithm replaces the reference-to-old-policy likelihood ratio with a reference-output distance penalty and replaces KL projection with MMD projection. The manuscript appropriately identifies the practical weights as a surrogate. Please clarify which properties of the original KL-regularized objective, if any, are claimed to carry over to the implemented update, and which derivations serve only as motivation.
7 comments
On the query in W2: Is the PerturbCellRL comparison fair given the omission of its original pathway reward?
PerturbCellRL uses DiffusionNFT as its likelihood-free update mechanism. We reconstructed its Pearson, RMSE, and DE-Spearman rewards from the June 2026 description, omitting the pathway verifier because its published annotations cover Norman and we lacked corresponding validated annotations for HepG2. We therefore label the baseline PerturbCellRL-inspired.
To address the reward mismatch, we additionally evaluated DiffusionNFT using our Gene-Effect and Pathway-Enrichment reward. We compare this new result with the original manuscript’s reconstructed-3R and MMD-PO results on HepG2 below.
| Method | EF@1% | EF@5% | Rank AnDCG@1% | Rank AnDCG@5% | Geo AnDCG@1% | Geo AnDCG@5% |
|---|---|---|---|---|---|---|
| DiffusionNFT + reconstructed 3R | 3.900 | 2.799 | 0.143 | 0.099 | 0.184 | 0.162 |
| DiffusionNFT + our reward | 1.510 | 1.920 | 0.051 | 0.062 | 0.122 | 0.119 |
| MMD-PO + our reward | 6.565 | 3.015 | 0.165 | 0.120 | 0.178 | 0.169 |
MMD-PO achieves higher point estimates than DiffusionNFT with the same reward across all six endpoints, supporting an advantage over this closely related likelihood-free alternative in the evaluated setting. We will revise the manuscript to clarify the baseline construction and include this additional comparison using the same reward.
Thanks for the Response
Thank you for the additional experiments. A few points remain:
-
Inconsistent numbers. The Wasserstein/Sinkhorn results in this thread and in the 9MxJ thread have identical AnDCG values but different EF values (e.g., Wasserstein EF@1% 7.240 vs 5.78; Sinkhorn 6.150 vs 4.97). Please clarify which values are correct. I note that 8.93 ± 1.45 is the five-inference-seed mean; please state whether 10.30 and the new baselines are single-inference-seed results or averages over multiple inference seeds.
-
Attribution to MMD. Mean L2 (reported to 9MxJ) recovers 70–89% of the improvement over SFT initialization across the six metrics. MMD has higher point estimates on all six, but without matched uncertainty estimates it is unclear whether its additional gains are robust. Could you report mean ± SD for all objectives using the same inference seeds and evaluation protocol, and confirm whether the candidate populations and target weights were held fixed across objectives?
-
Direct task-loss baseline. The best Huber configuration (EF@1% 3.13) is well below continued SFT (5.37). Please report the coefficient range, a λ = 0 control, and how the configuration was selected. This experiment also covers only the Gene-Effect term; a differentiable pathway surrogate, as suggested in my review, has not been evaluated.
-
DiffusionNFT. Which backbone and budget were used (MMD-PO EF@1% is 6.565 here)? When switching to your reward (EF@1% 3.900 → 1.510), were the reward scale and optimization hyperparameters re-tuned, and how?
I will reconsider my score after these clarifications.
Thank you very much for your positive comments and appreciation of our novelty, experiments and presentation. Below are our responses to the specific query raised.
Weakness 1: Can the gains be obtained by directly adding the biological task losses to the native supervised objective?
Thank you for your thoughtful comment. As requested, we added the Gene-Effect Huber loss to the native supervised objective and swept its coefficient. Below, we report the setting with the highest EF@1% in the sweep, alongside the original supervised-objective and reward ablations. The results are updated in Table 2.
The direct Huber-loss baseline does not beat the performance of MMD-PO. These results support the benefit of reward-weighted population fitting over the tested direct-supervision formulation.
For the pathway component, GSEA enrichment depends on a hard ordering of genes. Standard backpropagation does not provide gradients through changes in these discrete rank assignments. Weighted GSEA can retain gradients through score magnitudes within a fixed ordering, but would not be the same calculation as the original enrichment reward. Therefore directly adding the pathway reward as a loss won’t be suitable for our cases.
Weakness 2 and Question 1: Can you hold the reward-weighted candidate populations fixed and compare MMD with alternative distribution-matching or weighted-fitting objectives?
Thank you for your insightful question. To address this concern, we added comparisons replacing the outer MMD with Wasserstein distance and Sinkhorn matching. With the same initialization and optimization budget, we changed the MMD with other distribution matching functions:
MMD achieves the highest reported performance across all six metrics. These results support a contribution from the MMD fitting objective beyond candidate generation and reward weighting. We will include the transport costs, Sinkhorn regularization, and optimization details in the revision.