- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies Cross-Modal Drifting (CMD) which is proposed for conditional generation from paired cross-modal data where each source observation usually has only one observed target. As a result, the empirical conditional is a point mass. The paper constructs a richer local proxy and uses fixed source and target representations to retrieve nearby source observations and then assigns Gaussian distance weights. Those weights are transferred to the paired targets. A conditional generator is then trained with a kernel-gradient drifting field against this proxy. At inference, CMD generates a sample with one network evaluation and needs neither neighborhood retrieval nor field computation. The paper proves identifiability relative to the supplied proxy. Experiments cover a synthetic conditional circle, a cat-to-dog sketch task, BMMC CITE-seq RNA-to-protein generation, and HEST-IDC histology-to-gene-expression generation. CMD achieves the lowest mean Energy Score on both biological benchmarks and remains competitive as synthetic ambient dimension grows. CMD has higher training cost and its distributional gains do not consistently translate into better Pearson correlation.
Critical strengths
-
The paper identifies a real obstacle in conditional generative modeling. This is especially important in biology, where a CITE-seq cell yields one RNA and one protein profile, and a spatial transcriptomics spot pairs one histology patch with one expression readout.
-
CMD’s proxy is conceptually clean. It avoids storing or modeling a full conditional density and avoids assuming a parametric form. The use of Kish effective sample size to make MM comparable across queries is a suitable design choice.
-
CMD builds on Kernel-Gradient Drifting, where the drift field is the difference between the scores of kernel-smoothed target and generator distributions. This gives the method an attractive-repulsive field.
-
The optional CDC smoothing replaces point masses at paired targets with local Gaussian components estimated from target-space neighborhoods. This is a sensible way to inject local target geometry without changing the source-derived mixture weights.
-
CMD requires only one neural function evaluation at inference. Once trained, the generator maps noise and a source representation to a target sample without neighborhood retrieval, drift computation, or iterative sampling. This is a real advantage over flow and consistency method.
-
The component study separates reference drifting, KGD, CDC, the source-neighborhood proxy, and their combinations. This makes it possible to see which component drives gains.
Critical weaknesses
-
CMD assumes that nearby source representations have similar conditional target distributions. This is the central statistical assumption, but the paper does not provide a diagnostic for when it holds. The sketch task is designed so that nearby points on different strokes can have different conditionals, yet the paper does not isolate how much of CMD's error comes from neighborhood mixing versus generator fitting. On real data, source representations are fixed PCA or pretrained embeddings, and there is no check that their local neighborhoods preserve target-relevant structure.
-
CMD uses fixed source and target representation maps such as PCA for BMMC. The generator never adapts these representations. This simplifies training and makes the proxy stable, but it also means the method inherits any information loss or bias in the representations. If the source representation discards a factor that predicts target variability, no amount of proxy weighting can recover it.
-
CMD posts the largest training total on HEST-IDC and, after CDC-FM, on BMMC. The proxy positives make its update the most expensive in the drift family. The paper notes that KGD-proxy is cheaper when CDC's gain is small, but the main method still carries a substantial training burden. For biological applications with large datasets or many folds, this cost may be prohibitive. The paper does not establish a clear cost-quality or show that CMD's ES gains justify the added training time in downstream terms. A cheaper approximation may be preferable in many settings.
-
The evaluation covers two synthetic tasks and two biological benchmarks. There is no evaluation on image-text, audio-text, multimodal medical imaging, or other paired modalities. The biological tasks are both expression-prediction problems, and HEST uses one training seed per fold. The paper does not test robustness to missing modalities, noisy pairs, batch effects, or dataset shift. It also does not compare with strong conditional generative baselines beyond the flow/consistency family, such as conditional normalizing flows or conditional GANs.
-
CMD involves several choices and the paper fixes many of these from prior work or inner validation, but does not provide a systematic sensitivity analysis.
-
The code does not seem available which makes judgement about reproducing the results challenging.
Questions for authors
-
hen does the source-neighborhood assumption actually hold, and can it be diagnosed before training? CMD's proxy is only as good as the claim that nearby source representations have similar conditional target distributions.
-
Do CMD's distributional gains translate into downstream biological utility? ES improves, but Pearson correlation does not consistently improve and sometimes favors KGD or MeanFlow. Does better conditional calibration lead to better imputation, differential expression, cell-state recovery, or spatial prediction? The paper should connect ES gains to concrete downstream tasks, not only to a proper scoring rule.
-
hyperparameters from prior work or inner validation. How much do ES and Pearson change across reasonable ranges of all hyperparameters? Is the method stable across seeds and folds or do the gains depend on favorable calibration? A sensitivity study would clarify robustness.
6 comments
Q1
When does the source-neighborhood assumption hold, and can it be diagnosed before training?
We agree that this is the central statistical assumption (Section 3.1). It cannot be verified directly from single-observation data, because testing whether two nearby sources share a conditional distribution would require repeated targets per source. We therefore provide a direct measurement on the sketch task, where the conditional is known, and predictive checks that are available on real data before any generator is trained. These are the checks the current procedure already provides; the conclusion names dedicated diagnostics of proxy quality as a direction for further work.
Sketch: measuring neighborhood mixing. Appendix B.8 notes that nearby sources on different strokes can have different conditionals, and that the reported results alone do not separate neighborhood mixing from generator fitting. To complement that discussion, we computed for every held-out source the fraction of its proxy weight assigned to training sources on a different stroke, at the selected breadth (, three replicates, 768 held-out sources). The mean cross-stroke weight is 2.8% ± 0.2%, and about 97% of the 16 nearest training sources lie on the query's stroke. We then split the held-out sources into three groups by this fraction, using cut-offs of 10% and 30% chosen before inspecting the results, and report the mean conditional within each group. is computed per source between a method's generated samples and samples drawn from the true conditional:
| Cross-stroke proxy weight | Sources | Oracle sampler | Direct proxy | KGD + proxy + CDC (E) | FM, 50 NFE |
|---|---|---|---|---|---|
| At most 10% | 705 (91.8%) | 0.024 | 0.056 | 0.055 | 0.071 |
| 10% to 30% | 33 (4.3%) | 0.020 | 0.244 | 0.408 | 0.485 |
| More than 30% | 30 (3.9%) | 0.021 | 0.512 | 0.598 | 0.605 |
The first method column scores an independent sample from the true conditional in the same way as a method; it is the lowest attainable at this sample size. Every method remains more than twice this floor, including on the sources with nearly pure neighborhoods.
On the 92% of sources with nearly pure neighborhoods, the trained generator matches the proxy it was trained against (0.055 vs 0.056). Where neighborhoods mix strokes, the proxy's is roughly 4 and 9 times higher, and the generator's is higher still. FM, which uses no neighborhoods, degrades on the same sources, so they are hard for every method, and we do not attribute the generator's error there to mixing alone.
Real data: checks available before generator training. The proxy is itself a predictive distribution, so its inner-validation ES can be computed for every candidate breadth without training a generator. This is the criterion that selects (Section 3.2, Appendix A.1). The full curves are:
| 4 | 8 | 16 | 32 | 64 | 128 | |
|---|---|---|---|---|---|---|
| BMMC | 6.800 | 6.156 | 5.866 | 5.755 | 5.735 | 5.765 |
| HEST-IDC (mean of four folds) | 3.349 | 3.055 | 2.945 | 2.930 | 2.965 | 3.028 |
Both curves have an interior minimum. Very narrow neighborhoods are 13–19% worse than the selected breadth. The informative part is what happens beyond the selected breadth. If distance in the fixed representation were unrelated to the target, far neighbors would be as useful as near ones, and adding them would not worsen the score. Instead the score worsens as the neighborhood grows past the selected breadth, by 0.5% at on BMMC and, in every HEST-IDC fold, by 0.9–1.6% at and 2.7–4.4% at . Nearer sources therefore predict the target better than farther ones.
As a second check, the selected proxy can be compared with trained baselines before committing to generator training (Section 4.5, Table 3). These are held-out test scores for the same proxy at the selected . The scores in the table above are inner-validation scores, computed on queries held out at random from the training data, so the two are not comparable in magnitude. Sampled directly, the proxy attains held-out ES 6.712 on BMMC, lower than KGD trained on single paired targets (6.855) and all six competing objectives (6.938 or higher). On HEST-IDC it attains 6.555, lower than KGD (6.620) and five of the six competing objectives; CDC-FM at 50 NFE scores 6.528.
These are diagnostics of predictive usefulness, not guarantees that every neighborhood preserves the conditional law. As the sketch task shows, a favorable aggregate can coexist with a minority of mixed neighborhoods. Regarding the fixed representations, we agree that the proxy cannot recover information that the representation discards. Two remarks put this in context. All methods in our comparison are conditioned on the same fixed representations (Appendix B.1), so none of them has access to the discarded information. And the checks above test whether neighbors in the given representation predict the target; they cannot tell whether a different representation would do better. Section 3.1 states that the quality of the proxy depends on how well the source representation preserves target-relevant structure, and the conclusion names this as a limitation, with learning the representations jointly with the generator as a direction for further work.
We thank the reviewer for their careful reading and constructive feedback. We appreciate the opportunity to clarify CMD’s assumptions, discuss its downstream utility, and address its sensitivity to methodological choices. We address the three questions separately.