- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper introduces a generative model for single-cell data, which selects the dimensionality of the latent space by prior-fitting.
Critical strengths
Overall the paper is well-written and tackles an interesting problem.
Critical weaknesses
I am asking reviewers to assess two critical weaknesses:
W1: Relevancy of the problem setting:
In simple-terms, the procedure the authors propose seems to pick the top-k most informative latent dimensions. This dimensionality-reduction step is part of many real-world single-cell pipelines, and other literature has looked at automating this step with different heuristics, most prominently the 'elbow plot'. For different heuristics, see e.g.: Haotian Zhuang et al., findPC: An R package to automatically select the number of principal components in single-cell analysis, Bioinformatics 2022 How do these heuristics compare to the approach of the paper?
W2: Unrealistic benchmarking assumptions:
The authors benchmark with a ground truth number of cluster labels 'k'. This ground truth is not available in real-life scenarios, and changes the goal of the embedding. When analysing new single-cell data, biologists need to a) first define the available clusters in the data, b) assigning cell identities to them. A good embedding therefore supports both: selecting the number of clusters and cluster assignment. Knowing the ground truth number of clusters changes this task.
Minor comments:
- The paper sometimes contains surprising formulations in the appendix, e.g. L1153
no per-dataset configuration file overrides any of them
Questions for authors
- See Weakness 1: How were the dimensionalities for the datasets picked for the benchmarks? How does the method perform against established heuristics for picking the dimensionality?
- See Weakness 2: How does not-specifing
kin the benchmark impact performance? - What is the biological relevancy of the scenario in L373 (removing canonical markers):
- What is the implication of the theoretical guarantees?