- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes responder-weighted flow matching for generating cell morphology images under heterogeneous perturbation responses. It estimates per-cell response scores using a classifier cross-fitted over wells, then weights the flow-matching loss to emphasize likely responders. A controlled dilution benchmark shows recovery of effects lost by standard training, while tests on BBBC021, RxRx1, and JUMP show gains on some datasets but mixed results on JUMP.
Critical strengths
1 The paper gives a clear derivation of mixture-induced effect shrinkage and shows how calibrated response weights change the flow-matching target.
2 The dilution benchmark has known responder labels, and the real-data evaluation includes well-level cross-fitting, multiple metrics, ablations, and the negative JUMP result.
Critical weaknesses
1 The responder model assumes non-responders follow the same distribution as same-plate controls. The paper’s own diagnostics (Table 4) show that low-scoring treated cells remain distinguishable from controls on BBBC021 and RxRx1. The authors consequently interpret the weights as emphasizing more affected cells rather than identifying a clean responder subset. Thus, real-data results support a reweighting heuristic, but do not validate the estimated responder fractions or posterior probabilities as such.
2 The effect of the weighting exponent is dataset-dependent. On JUMP, (\gamma=1) raises the energy distance to the full treated population to 5.88 times the unweighted value, while (\gamma=0.25) improves it (Table 17). The paper does not establish how to choose (\gamma), or whether the intended target for a new dataset is the responder population or the full treated population.
Questions for authors
Given the Table 4 diagnostics, should (\hat{\rho}) and (s(x)) on BBBC021 and RxRx1 be interpreted as responder probabilities, or only as scores that rank cells by perturbation effect? Please clarify which real-data claims rely on each interpretation.
How should a practitioner select (\gamma) for a new dataset without using test-set outcomes, and how should they decide whether to model responders alone or the full treated-cell mixture?
3 comments
Weakness 2 and Question 2.
The paper did not say how to choose the target or , and we have added a two-step procedure to Section 5.5. First, the target follows from the question. It is the whole treated population when the generated cells should stand in for a well, for example to predict well-level profiles, and the responders when they should show what a perturbation does to the cells it affects, for example to identify its mechanism. Second, is chosen on validation wells with a metric that matches the target: the energy distance to all treated cells for the population, and MoA accuracy on BBBC021, top-5 accuracy on RxRx1 or mAP on JUMP for the responders. So no test outcomes are needed.
To test the second step, we split the evaluation wells 20 times at random into two halves, chose on one half, and checked whether it was also the best on the other half, so the choice never sees the wells it is checked on. The candidates are of 0, 0.25, 0.5, 1 and 2 on BBBC021 and RxRx1, and 0, 0.25 and 1 on JUMP, where two seeds give the same counts. The table below counts the splits, and we have added it to Appendix E.
| Dataset | Validation metric | chosen | Also best on the other half |
|---|---|---|---|
| BBBC021 | Energy distance to all treated | 2 (19), 0.5 (1) | 18 |
| MoA accuracy | 2 (20) | 20 | |
| RxRx1 | Energy distance to all treated | 1 (20) | 20 |
| Top-5 accuracy | 1 (14), 0.5 (3), 0.25 (3) | 7 | |
| JUMP | Energy distance to all treated | 0.25 (20) | 20 |
| mAP | 1 (20) | 20 |
Validation wells help to pick reliably. For the population, they pick mild weighting on JUMP, where few cells respond, and strong weighting on BBBC021 and RxRx1, where most do. For the responders, they pick on JUMP (mAP) and on BBBC021 (MoA accuracy). With these metrics, the choice misses the best in only 2 of 100 splits, losing about 1% of the gain over . The exception is top-5 accuracy on RxRx1, which is too noisy on half of the wells to rank from 0.25 to 2. There the choice matches the best in only 7 of 20 splits, but it never picks and keeps about half of the gain.
The response addressed my comments. I will keep my score at 3.
Weakness 1 and Question 1.
Thank you for your careful review and for recognizing the "clear derivation of mixture-induced effect shrinkage". Following your comments, the revised paper states more clearly how to interpret ρ^ and s on BBBC021 and RxRx1, and it now explains how to choose the target and γ.
We agree with your point. Our responder model assumes that each treated cell either responds or looks exactly like a control. On BBBC021 and RxRx1, even the treated cells with the lowest scores still differ slightly from controls (Table 4), so there the response is graded rather than all-or-none. In that case s is not the probability that a cell responded but a score that ranks cells by how strongly they differ from controls, and ρ^ counts the cells that differ clearly from controls. On JUMP the same check stays near 0.5, as the assumption predicts. We now state this in Section 3.2 and in the Limitations paragraph of Section 6.
Three statements need calibrated probabilities: that γ=1 learns exactly the responders (Section 3.4), that ρ^ is the share of responders (Section 5.1), and that mixture mode rebuilds the treated population (Section 5.5). The gains of weighting on real data need two properties of s, and neither requires it to be an exact probability: s must rank the cells well, and it must give control-like cells only the floor weight smin. The table below removes one property at a time. Shuffled weights put the weights of s on random cells of the same perturbation, which removes the ranking. The raw classifier output h ranks cells like s, but gives a control-like cell about half the weight of a clear responder when few cells respond. Values are mean ± standard error over three seeds, with one seed for shuffled weights on BBBC021.
Without the ranking, weighting keeps only part of the gain, and none of it on RxRx1. Without the floor, it keeps the gain on BBBC021, where almost all cells respond, but loses most of it on JUMP, where few do. So only the three statements above need calibrated probabilities. The gains on real data need just the ranking and the floor.