- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Vis2Cell-Bench as an evaluation benchmark to test whether spatial transcriptome super-resolution methods can recover single-cell level gene expression from aggregated spatial measurement data. The authors aggregated Xenium-sequenced cellular data from 87 datasets, simulated spatial measurement results similar to Visium, and compared seven methods under various spatial grid configurations, grid offsets and gene panel settings. The results indicate that gene-wise reconstruction metrics do not necessarily reflect whether cell-level structure is preserved. In addition, the paper proposes CRS and evaluates the relationship between CRS and annotation based NMI on a breast cancer dataset with cell annotations.
Critical strengths
1.Based on experimentally measured Xenium data, the benchmark constructs simulated spatial measurements while retaining the original cell-level data as references. This enables evaluation of methods’ ability to recover cell-level expression.
2.The benchmark includes 87 datasets and compares seven methods across multiple grid, offset, and gene-panel settings. The results show that higher gene-wise reconstruction scores do not necessarily indicate better preservation of cell-level structure. This suggests that evaluations should consider both aggregate gene-expression reconstruction accuracy and cell-level organization.
Critical weaknesses
1.CRS is evaluated against cell-type annotations using only one breast-cancer dataset and one gene panel. Because this is the paper’s key validation of CRS as a measure of cell-level structure, the evidence does not establish whether the metric generalizes across datasets, tissues or gene panels.
2.The benchmark shows that gene-wise reconstruction metrics and cell-level structure metrics can disagree. The authors introduce CRS to quantify cell-level structure, but its evaluation does not convincingly establish that CRS provides a robust or practically useful signal or clarify how it should guide method assessment. Thus, the main insight remains largely descriptive, and the contribution beyond a large-scale comparison of existing methods is unclear.
Questions for authors
1.The article does not explain how to summarize the CRS at the spot level into comparative results, nor does it explain how to handle spots that cannot calculate CRS. Because the fact that different methods may exclude different spots and affect the ranking of the methods, can the author explain the summary method and report the number of spots excluded under each method and setting?
2.The CRS-NMI correlation is calculated based on a combination of 84 methods, grids, and offsets. Are these combinations from the same annotated sample? If so, would it be possible that they are 84 independent biological replicates, and would their interpretability be weakened as a result?
3.CRS is validated against annotation-based NMI on only one breast-cancer dataset and one gene panel, providing limited evidence for its generalizability. Do the authors have validation results from additional annotated datasets or gene panels?
4.The appendix provides a brief introduction to the details of each method. Can the author provide key parameters, preprocessing steps, etc. for each method? And do different methods use external information such as tissue images, reference single-cell data, cell types, etc? Is the comparison of these models fair?