- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a latent OT-CFM model whose velocity field is also conditioned on a mean-pooled embedding of the source population and on a transition descriptor. The method is evaluated on held-out donors in iPSC differentiation (Cuomo), unseen Norman Perturb-seq combinations, and unseen sci-Plex3 compounds.
Critical strengths
The evaluation is overall well done and presents results in a honest way. Experiments state clearly what the model can see of the targets, and they separates swapped-context controls from matched retraining. A single formulation covers temporal, genetic and chemical tasks. The synthetic ambiguity experiment is clean.
Critical weaknesses
I have found the paper to be overall a fairly hard read despite the simplicity and limited methodological novelty (in a ML sense) of the paper. The introduction is in that sense quite vague and should do a better job of highlighting the main differentiator, methodolofically, compared to previous work. More precisely:
- Conditioning a flow on an embedding of the initial population is the central idea of Meta Flow Matching (Atanackovic et al., 2024), which is neither cited nor compared against. Schrodinger-bridge and multi-marginal trajectory methods (e.g. SF2M) are also missing. A very similar idea (although not flow based) is proposed in "Supervised Training of Conditional Monge Maps" by Bunne et al. 2022.
- The theory in App. B.5 reduces to the standard fact that conditioning on more variables lowers the Bayes risk. It does not show that real data contain this ambiguity, or that mean pooling keeps the relevant information.
- On real data, the benefit of source context is small or mixed. In the matched retraining (Table S13), Energy is 0.135 vs 0.139, and r and DE@100 favor the model without context. In the genetic task the source panel is fixed, so the context is constant and the model reduces to conditional OT-CFM with auxiliary losses.
- The reported standard deviations vary only the test seed. The auxiliary losses directly optimize the evaluation metrics. The pEnergy of the CellFlow adapter (9.22) suggests a broken baseline.
Questions for authors
- How does the method differ from Meta Flow Matching, and how do the two compare empirically?
- How much do results vary across training seeds?
- How does the model perform without the auxiliary losses?
- Did you try richer set encoders than a mean?
- What batch sizes did you use for Sinkhorn? Where do the different epsilon/num iteration settings come from across experiments and why did you change them?
minor:
- "Because sequencing destroys measured cells, training provides unpaired source and target populations rather than trajectories of the same cells (Schiebinger et al., 2019)" -> training provides, in what sense is "training" the subject of the verb "provides"?
3 comments
Thanks for your rebuttal
Thanks for your experiments.
- I find it a bit discouraging to see on these experiments (W1) that your OT variant has higher energy & MMD than random (independent) coupling. I feel this "kills" a part of your story when OT is no longer useful, or even detrimental.
- I am a bit skeptical of your Sinkhorn settings. Why set #max iterations (perhaps your solver did not converge?) rather a convergence threshold (as usual). How did you set your batch sizes? It feels that n of the order of 64~128 is way too small to be robust. In particular you wrote "No common tuning rule was documented", what does this mean? documented by whom?
Since the OT part is a bit flawed and incomplete as it stands, yet takes a prominent role in your framing, I suggest that you revisit these aspects and dive more into whether changing key quantities (batch sizes, preferably increased or even max-ed to total data sample size) or epsilon could reshape your results. You could for instance report KL of your coupling to independent coupling as a natural metric to reflect the "peakyness" of your couplings, rather than just put the table you provided that has been tuned... in "some" way? (I could not understand what you meant and "No common tuning rule was documented" seems very LLM-ish)
Overall I am grateful to the authors for their improvements and efforts, but I think the paper still needs a solid round to (a) have results that are more consistent with the story line (i.e. where minibatch OT consistently wins, and we understand at what regimes) and (b) do the required ablations and keep on turning the screw where needed (e.g. CellFlow).
Thank you for asking whether a larger pool of cells would help OT. In the temporal task, increasing the pool from 64 to 128 cells per side gave couplings closer to those computed from all available cells in each record. It did not improve prediction.
Six training records had at least 128 cells on both sides. We used the same six records at both pool sizes, fixed ε at 0.03, and sampled 64 training pairs per record. To compare the couplings, we first computed a reference using all available cells in each record. We took the part of that reference corresponding to the sampled cells and renormalized it. The mean total variation distance from this reference fell from 0.1745 at 64 cells per side to 0.0967 at 128. Eleven of the twelve training runs finished. For the two seeds with all four settings complete, increasing the pool raised OT’s mean test Energy from 0.1315 to 0.1371 and its mean test MMD from 0.3521 to 0.3664. Lower values are better for both metrics.
Sinkhorn settings. Each temporal training update sampled 256 records. We solved a separate coupling for each record using 64 source cells, 64 target cells, ε = 0.03, and 200 iterations. Our earlier phrase “no common tuning rule was documented” referred to our own experiment records, which list the settings used for each task. We will report the number of cells per coupling, ε, and the number of Sinkhorn iterations for each task in the main text.
To check whether 200 iterations were enough, we replayed the first training batch for each of three seeds (768 couplings) and drew 300 more samples at three frozen checkpoints. At 200 iterations, 1,044 of the 1,068 couplings met a marginal L1 tolerance of 10⁻⁴ on both sides. At 500 iterations, all 1,068 met it. Even after extending the runs to 2,000 iterations, the total variation distance from the 200-iteration solution was below 0.000233 for every coupling we checked.
We also tested whether more iterations improved prediction. We trained two models with seed 42 using the same initial weights and cell samples, then evaluated them on the same 15 held-out donors. Increasing from 200 to 500 iterations raised test Energy from 0.13141 to 0.13219 and test MMD from 0.34759 to 0.35763. The extra iterations improved numerical accuracy in the coupling check, but neither test metric improved in this run.
Effect of ε. With 64 cells per side, mean KL divergence from independent coupling fell from 2.06 at ε = 0.015 to 1.14 at ε = 0.03 and 0.46 at ε = 0.06. We tested all three values in matched training runs with seed 42 and 200 Sinkhorn iterations. At the original ε = 0.03, test Energy was 0.13141 and test MMD was 0.34759. Halving ε to 0.015 raised Energy to 0.13550 and MMD to 0.35633. Doubling ε to 0.06 raised Energy to 0.13234 and MMD to 0.35759. These results support keeping ε = 0.03 for this task. Halving or doubling it changed the couplings, with modest increases in test error for this seed: 0.7%–3.1% for Energy and 2.5%–2.9% for MMD.
Contribution. MFM models how cell velocity can depend on the starting population, and SCALE uses a population summary to predict the treated endpoint. Our contribution is to quantify the cost of omitting source information in velocity prediction and to test separately whether a model uses that information and whether it improves predictions. With pairings and paths fixed, two source populations can have different mean velocities at the same cell state, flow time, and transition. If this occurs with positive probability, omitting the source raises the minimum achievable mean squared velocity error (Section 3.2 and Appendix B.5).
Both models in the pairing comparison received the same source summary. We tested the role of the summary in two separate experiments. Swapping donor summaries changed predictions while the model weights and starting cells were held fixed. In matched retraining, adding the summary reduced Day 1 cell-state composition error in all three seeds, by 4.77 percentage on average. Effect correlation was higher without the summary.
CellFlow. We checked whether the 50-component PCA representation in our CellFlow adapter could account for the high projected Energy. Compressing and reconstructing the measured target cells alone gave a projected Energy of 9.4106, before any prediction. We then retrained the adapter on all 2,000 genes, using the same test cells and evaluation projection. Its projected Energy fell from 9.2228 to 0.2874, while its effect correlation fell from 0.5608 to 0.0805.
Thank you for the careful review.
W1/Q1. Meta Flow Matching (MFM). Atanackovic et al. condition a flow on the starting population. Both methods can predict intermediate states. MFM aggregates across neighbours before pooling. TempoBridge averages individual cell embeddings and uses that summary throughout the flow.
Our donor swaps test whether a fitted model uses source context (S14). Matched retraining tests whether providing that context improves prediction (S13).
The Cuomo pipeline is our adaptation of the authors' released TrellisMFM graph encoder and velocity network. On a synthetic batch, our adapter reproduced the embeddings and integration obtained by calling their networks directly. We matched source cells, expression encoder initialization, auxiliary losses and checkpoint selection by validation MMD. We trained on Day 0 to Day 3 and tested on Days 1 and 2 in 15 held-out donors (2,009 genes). T2 and MFM used random pairing within donors.
Both TempoBridge variants had lower mean Energy (our primary metric) than MFM. Under matched random pairing, T2 had lower Energy and MFM had higher effect correlation. Within TempoBridge, S4 favoured OT at fixed checkpoints (0.1291 vs 0.1446). In the comparison above, random pairing had lower mean Energy (0.1366 vs 0.1383).
W2/Q4. Theory and pooling. Appendix B.5 applies the squared-error Bayes risk identity. With the coupling and paths fixed, source information lowers the minimum error if mean velocity still depends on the source given cell state, time and condition. Table 4 gives one example. How often this occurs in real data, and whether the mean retains the signal, remain open. The MFM and SCALE-style comparisons use richer encoders but also change the predictor, so neither isolates pooling.
W3. Source context. In S14, we changed only the donor summary. For 12 of 15 donors, Energy with their own summary was lower than the average obtained with the other donors' summaries.
In S13, with three training seeds each, mean Energy (SD) was 0.1350 (0.0051) with source context and 0.1393 (0.0133) without it. Effect correlation and DE@100 favoured training without context. A classifier fitted on training donors showed that context reduced stage-proportion error by 4.7739 percentage points on Day 1 and 1.2593 on Day 2, though Day 1 error remained high at 89.5173 points. The genetic task uses the same 128 controls throughout, so its source panel is fixed.
W4/Q3. Genetic auxiliary losses. In Seen 2/2 both genes appeared in training, but their combination was new. Some losses directly supervise quantities we evaluate, including mean expression. We removed all seven population losses together, keeping flow matching and both reconstruction losses.
Removing the losses increased both mean-expression error (Mean L2) and distributional distance (Energy), and also raised both errors on the other two splits with OT. In Seen 2/2, OT gave lower errors than random pairing with the losses off. Under full supervision the errors were similar.
CellFlow baseline. Compressing measured target cells to 50 principal components and reconstructing them gave a projected Energy of 9.4106 before prediction. We retrained our adapter on all 2,000 genes, keeping the test cells and evaluation projection fixed. Projected Energy fell from 9.2228 to 0.2874 (TempoBridge, 0.2123), while effect r fell from 0.5608 to 0.0805. The original adapter overstated our distributional advantage.
Q5. Sinkhorn settings. Each plan pairs cells within one record. For biological tasks we divided squared latent distances by their within-record median.
We kept each task's original settings. No common tuning rule was documented. The earlier four-GPU temporal setup specified 1,024 records per update.
Manuscript changes. We will refocus the Introduction on when source context helps and credit MFM, SF²M and conditional Monge maps. We will add these comparisons and settings, correct the CellFlow row, and revise the flagged sentence to read "Because single-cell RNA sequencing destroys each assayed cell, we observe unpaired source and target populations."