- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes optimal-transport t-SNE (OT-tSNE). Datasets collected under different conditions are affected by batch effects, which make joint visualization of the data difficult. To address this problem, the proposed method provides a methodology that generates well-integrated visualizations across batches while preserving the structure of each individual batch. The authors evaluate the visualization of seven datasets containing batch effects in two dimensions and demonstrate the effectiveness of the proposed method.
Critical strengths
As stated in the contributions of this paper, the primary strength of this work lies in proposing OT-tSNE, the first combination of optimal transport and neighbor embedding methods for the integrated visualization of data with batch effects. The paper also shows that while conventional nearest-neighbor-based methods can degrade even when the batch effect is orthogonal to the data, OT-tSNE is not affected by it. Furthermore, through a large-scale benchmark for visualizing data with batch effects, this paper demonstrates that OT-tSNE achieves high performance, particularly in batch mixing.
Critical weaknesses
A more in-depth discussion of the validity of the parameters used in the proposed method is desirable. For example, the rationale for choosing the values of the parameters described in the Implementation details, including setting the number of neighbors to 15, is not sufficiently explained. It would also be desirable to clarify how sensitively these parameters affect the results of the proposed method.
A theoretical comparison of the computational complexity of the proposed method with that of existing methods is desirable. Although the paper discusses runtime in the Limitations and Future Work section, runtime depends on the implementation and the computing environment. The reviewer considers it necessary to base the discussion on a comparison of theoretical computational complexity, rather than on whether the method can be accelerated through implementation.
The reviewer would like the authors to present the algorithm of the proposed method explicitly, for example in the form of pseudocode. This would clarify how the processing pipeline differs from that of existing methods and which steps, if removed, would reduce the method to existing work. As a result, it would also become easier to understand how the newly added steps contribute to the effectiveness of the method. In addition, it would be desirable for the description of the proposed method to include a theoretical explanation, through comparison with existing work, of why the proposed method can solve the problem.
OT-tSNE is currently formulated for two batches. However, since real-world data may involve three or more batches, the reviewer believes that the authors should present a concrete method for extending it to multiple batches and conduct experiments that reflect realistic problem settings. Note that this point is listed here as a limitation of the proposed method and does not lower the reviewer’s evaluation of this paper.
Questions for authors
The quantitative evaluation is based on rankings, and this paper states that this follows established practice in prior work. For the proposed method, the results sufficiently confirm its effectiveness. On the other hand, the reviewer would like the authors to explain whether the rankings among the compared existing methods can be considered valid from the perspective of the proposal in this paper.
How sensitively do the parameters of the proposed method affect accuracy? The reviewer would like to see results that clarify this. In addition, the reviewer would like the authors to clarify, in terms of the number of parameters, how much more complex the proposed method is compared with existing methods.
The paper states that OT-tSNE relies on balanced optimal transport, which forces every observation in each batch to be equally associated with the other batch. The reviewer would like the authors to clarify how much this constraint affects the results when applied to real-world data.
6 comments
Effect of marginal constraints when applied to real data
Thank you for this question! We believe our initial submission did not sufficiently emphasize that the cell-type compositions in the seven benchmarked datasets are not balanced (Figs. S1, S2). Despite the imbalances, our real data benchmark showed that OT-tSNE can achieve strong batch mixing and competitive cell-type preservation even with balanced marginal constraints. On Mouse Atlas, for example, B cells accounted for roughly 29% of one batch and 8% of the other (Fig. S1c). We identified Mouse Atlas as the dataset with the second largest compositional imbalance in the benchmark. Yet OT-tSNE had the strongest balance in batch mixing and bio preservation compared to other methods on Mouse Atlas (Fig. S5). We now extended our Section 5.2 and Conclusion to state these results more explicitly.
Extension to multiple batches
We are currently working on extending OT-tSNE for multibatch integration. For each pair of batches, we compute an OT plan and use it to construct the corresponding cross-batch affinity block. Within each batch, we compute affinities using either OT or kNN, or set the corresponding block to zero. We combine these blocks into a single affinity graph and sparsify it as in the two-batch method. A single t-SNE optimization on this graph then yields a joint embedding of all batches.
Preliminary results on two real single-cell datasets settings are promising. A broader benchmark is underway to assess how consistently the extension preserves cell types while mixing batches across multiple batches.
Computational complexity and hyperparameter complexity comparison
We are currently comparing OT-tSNE with the benchmarked methods in terms of computational complexity and the number of hyperparameters that require tuning. We plan to share this comparison during the discussion period.
Re: Official Comment by Authors
Thank you for submitting your comments. The reviewer has responded favorably to many of the revisions.
However, regarding the validity of the parameters, while the authors mention that they are using values commonly employed in other methods, it does not seem that they have gone into detail regarding why those values are appropriate for OT-tSNE. The reviewer would like to learn more about this point.
The reviewer also looks forward to the authors’ comments regarding the comparison of computational complexity and the complexity of the number of parameters.
Dear reviewer WXuz,
Thank you very much for the time invested in reading our paper and the detailed feedback you provided!
Validity of the chosen hyperparameters and sensitivity analysis
OT-tSNE has two main hyperparameters for affinity construction. The neighborhood size k controls graph sparsity. A small k can omit local connections, while a large k retains weaker affinities that can blur local structure. We chose k=15, following the neighborhood-size trade-off discussed for UMAP (McInnes et al., 2018). Overall, it is a standard default for neighborhood graphs in visualization, used in UMAP and NCVis (McInnes et al., 2018, Artemenkov & Panov, 2020).
For entropic regularization, we follow Landa et al. (2024) and set ϵ=rmedian(C), where C is the matrix of squared Euclidean distances between batches. Setting r=1 makes the regularization strength adapt to the cost scale of each dataset, which ''has been found robust against high-dimensional noise and theoretically appealing'' (Landa et al., 2024, Ding & Ma, 2023).
The sensitivity analysis we now added to Appendix E.1 supports our choice of defaults. When averaged across the seven datasets, the overall score improved by less than 0.01 relative to the default (k,r)=(15,1). We discuss how varying k and r affected batch mixing and biological preservation in our response to Reviewer WXuz.
Pseudocode
We added the pseudocode in the new Appendix C.1 and refer to it in line 187 of the revised manuscript. Thank you for suggesting this addition!
We fully agree that it is important to understand why OT-tSNE can help with batch integration. This was our aim in Section 5.1, where we compared OT-based affinities with the shared and mutual kNN graphs used by MultiMAP and Scanorama, respectively. We focused on perturbations orthogonal to the biological signal because this assumption also motivates other batch correction methods (e.g., Haghverdi et al., 2018). Eqs. 8 and 9 give a theoretical explanation for the advantage we observed for OT-tSNE in this setting. Specifically, the invariance of the OT plan to these perturbations helps explain why OT-tSNE's cross-batch affinities remained stable even under strong batch effects, while the quality of the cross-batch connections in the kNN graphs declined.
In the Introduction, we also explain our choice of OT by examining where cross-batch kNN graphs can fail. Fig. 1b illustrates a complementary case in which a constant shift along the signal direction changes cross-batch nearest neighbors, leading to a ''kissing problem'' in the visualization. The balanced OT plan remains invariant at fixed ϵ because the shift adds only row and column terms to the squared-distance matrix. The figure illustrates this effect in two dimensions, but the argument holds in any dimension. We then examined this case on MNIST by comparing visualizations from OT-tSNE and kNN-based methods. We plan to provide Jupyter notebooks illustrating both constant and heterogeneous batch effects, with perturbations either parallel or orthogonal to the signal subspace.
Interpretation of the rankings
The rankings summarize overall performance, batch mixing, and biological conservation across the seven datasets. The overall ordering reflects the trade-off between batch mixing and biological conservation evaluated in this paper, while the two category rankings show how methods compare on each objective separately. For example, LIGER 50D ranked ahead of MultiMAP on batch mixing, while MultiMAP ranked ahead on biological conservation. With both objectives weighted equally, MultiMAP placed higher overall (Fig. S4).
We revised the main text to clarify how we compute the ranks and added guidance on their interpretation in Appendix D.1. Since a rank difference does not convey the size of a performance difference, we also report the metric scores themselves. The pooled plots show how methods compare on both batch mixing and biological conservation (Fig. 4), and the per-dataset plots show how that comparison varies across datasets (Figs. S10, S11).