- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose CellRFT, a reinforcement fine-tuning framework for single-cell perturbation prediction. The goal is to address the mismatch between conventional training losses and biological evaluation metrics by using the latter as rewards in a policy-gradient optimisation procedure. The framework generates multiple candidate cell populations for each perturbation and assigns a shared reward to their constituent generation trajectories. To combine different biological objectives, the authors propose a hierarchical reward aggregation scheme that averages metrics within evaluation categories and subsequently across categories. The method is evaluated on the Replogle dataset using two variants of PerturbDiff. The authors report improvements in several differential-expression and perturbation-discrimination metrics and investigate the interactions between different rewards through a series of ablation studies.
Critical strengths
The mismatch between training objectives and downstream evaluation criteria is an important issue in single-cell perturbation modelling. Exploring whether these criteria can directly guide model optimisation is definitely a worthwhile direction.
Conceptually, extending policy-gradient optimisation to rewards computed on entire cell populations can be a reasonable approach to handling evaluation criteria that cannot be meaningfully computed for individual cells.
I also appreciated the experiments investigating how optimising individual metrics affects the others. The observed trade-offs, particularly the recall-related reward-hacking example, are interesting and could inform the design of evaluation objectives for perturbation models.
Critical weaknesses
The paper is generally readable, but I found the presentation somewhat repetitive. The motivation and the three main components of the framework are reiterated several times, while some important technical aspects of the actual optimisation procedure receive relatively limited explanation.
-
One of the core motivations for the approach is the potential non-differentiability of biological evaluation metrics. While this is true for metrics involving statistical significance thresholds, rankings, and discrete gene sets, it is less clear to me why reinforcement learning is preferable to differentiable approximations of these objectives -- e.g. expression fidelity, differential-expression effect recovery, and perturbation discrimination can be addressed through differentiable regression, ranking, or contrastive objectives. Moreover, population-level distribution matching is generally handled using established losses such as MMD or energy distance. The authors do not compare CellRFT with continued supervised fine-tuning using these objectives.
-
The experimental evaluation is restricted to one dataset and one train/test partition. Moreover, the manuscript does not report variability across random seeds, confidence intervals, or statistical significance tests. This is particularly concerning for the ablation studies, where small differences are used to support conclusions about reward composition and aggregation. For one, the advantage of hierarchical aggregation over flat averaging in Figure 4 is not particularly large. Given the stochastic nature of the generative model and the policy optimisation procedure, I believe repeated experiments and uncertainty estimates are essential to assess the reliability of the findings.
-
The authors repeatedly claim applicability across different pretrained models, but CellRFT is only applied to PerturbDiff, using two configurations of the same architecture. Although STATE, CellFlow, and other methods appear in the comparison, they are not used to assess the proposed fine-tuning framework.
-
I am not entirely convinced that the comparisons provide sufficient evidence for the effectiveness of the proposed framework relative to existing approaches. Several baseline methods were developed and trained for different objectives, and their reported performance may depend considerably on the training protocol and the evaluation setting. The manuscript would benefit from clarifying whether all baselines were evaluated under comparable conditions and whether additional adaptation was considered. More importantly, the authors do not include a compute-matched continued-training baseline.
Questions for authors
-
Q1: Can the authors compare CellRFT against compute-matched supervised fine-tuning using differentiable biological objectives?
-
Q2: Can the authors report results across multiple seeds (preferably different splits as well), with confidence intervals or standard deviations?
-
Q3: Can the authors evaluate CellRFT on additional datasets and generative backbones to support their claims of general applicability?
-
Q4: Can the authors clarify whether the competing methods were trained and evaluated under comparable conditions, and whether any task-specific adaptation or fine-tuning was performed?
I find the approach interesting, and I hope the authors can address these concerns and provide further evidence supporting their conclusions.