acceptodds
Under review as a conference paper at ICLR 2027

Vis2Cell-Bench: A Systematic Benchmark of Spatial Transcriptomics Super-Resolution

Abstract

Spatial transcriptomics super-resolution aims to infer cell-level gene expression from spatial measurements that aggregate signals across multiple cells, enabling downstream analyses of cell types and states. However, agreement with coarse observations does not establish whether reconstructed profiles preserve the cell-specific transcriptomic variation required for such analyses. We introduce Vis2Cell-Bench, a benchmark built from measured Xenium data for direct evaluation of cell-level recovery. Measured cellular profiles are spatially aggregated into controlled pseudo-Visium observations while being retained as ground truth. Across 87 datasets spanning 13 tissue and organ categories, four spatial partitions and three spatial offsets yield 1,044 benchmark instances for comparing seven methods under a common evaluation protocol. We evaluate two complementary aspects of recovery: gene-expression reconstruction and preservation of cell-specific information across gene panels and spatial partitions. These capabilities are not necessarily aligned: strong gene-wise reconstruction does not consistently translate into faithful recovery of cellular organization, and relative method performance varies across panels and partitions. In an annotated breast cancer dataset, high gene-wise correlation can coexist with limited cell-type separation. To quantify cellular recovery more directly, we introduce the Cellular Recovery Score (CRS), which combines matched-cell expression fidelity with preservation of within-spot transcriptomic relationships without requiring cell-type labels. Vis2Cell-Bench thus provides a unified framework for distinguishing expression reconstruction from preservation of cell-specific information in spatial transcriptomics super-resolution.

Primary area:
datasets and benchmarks
Keywords:
spatial transcriptomics, super-resolution, single-cell reconstruction, benchmark, computational biology

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rSiWOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Vis2Cell-Bench as an evaluation benchmark to test whether spatial transcriptome super-resolution methods can recover single-cell level gene expression from aggregated spatial measurement data. The authors aggregated Xenium-sequenced cellular data from 87 datasets, simulated spatial measurement results similar to Visium, and compared seven methods under various spatial grid configurations, grid offsets and gene panel settings. The results indicate that gene-wise reconstruction metrics do not necessarily reflect whether cell-level structure is preserved. In addition, the paper proposes CRS and evaluates the relationship between CRS and annotation based NMI on a breast cancer dataset with cell annotations.

Critical strengths

1.Based on experimentally measured Xenium data, the benchmark constructs simulated spatial measurements while retaining the original cell-level data as references. This enables evaluation of methods’ ability to recover cell-level expression.

2.The benchmark includes 87 datasets and compares seven methods across multiple grid, offset, and gene-panel settings. The results show that higher gene-wise reconstruction scores do not necessarily indicate better preservation of cell-level structure. This suggests that evaluations should consider both aggregate gene-expression reconstruction accuracy and cell-level organization.

Critical weaknesses

1.CRS is evaluated against cell-type annotations using only one breast-cancer dataset and one gene panel. Because this is the paper’s key validation of CRS as a measure of cell-level structure, the evidence does not establish whether the metric generalizes across datasets, tissues or gene panels.

2.The benchmark shows that gene-wise reconstruction metrics and cell-level structure metrics can disagree. The authors introduce CRS to quantify cell-level structure, but its evaluation does not convincingly establish that CRS provides a robust or practically useful signal or clarify how it should guide method assessment. Thus, the main insight remains largely descriptive, and the contribution beyond a large-scale comparison of existing methods is unclear.

Questions for authors

1.The article does not explain how to summarize the CRS at the spot level into comparative results, nor does it explain how to handle spots that cannot calculate CRS. Because the fact that different methods may exclude different spots and affect the ranking of the methods, can the author explain the summary method and report the number of spots excluded under each method and setting?

2.The CRS-NMI correlation is calculated based on a combination of 84 methods, grids, and offsets. Are these combinations from the same annotated sample? If so, would it be possible that they are 84 independent biological replicates, and would their interpretability be weakened as a result?

3.CRS is validated against annotation-based NMI on only one breast-cancer dataset and one gene panel, providing limited evidence for its generalizability. Do the authors have validation results from additional annotated datasets or gene panels?

4.The appendix provides a brief introduction to the details of each method. Can the author provide key parameters, preprocessing steps, etc. for each method? And do different methods use external information such as tissue images, reference single-cell data, cell types, etc? Is the comparison of these models fair?

Reviewer_4vHBOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a benchmark for evaluating cell-level gene expression recovery in ST super-resolution (Vis2Cell-Bench). The authors construct pseudo-Visium data by aggregating Xenium cellular expression and retain the original cell-level profiles as ground truth. The experiments cover 87 datasets (1,044 benchmark instances), comparing seven methods. The results show that higher gene-wise reconstruction accuracy does not necessarily mean better preservation of cellular expression structure. The authors also propose the CRS, which combines matched-cell expression fidelity with preservation of within-spot transcriptomic relationships. The metric is further evaluated on an annotated breast cancer dataset.

Critical strengths

This paper addresses the recovery of cell-specific expression information in ST super-resolution. Most work in this area still judges reconstruction by agreement with measured expression. PCC and RMSE are useful for that purpose, but they say little about whether cell-to-cell differences survive or whether the reconstructed data remain usable downstream. The motivation is clear, and CRS is a sensible attempt to measure something that these standard metrics miss.

The experiments cover a large number of datasets from different organs and tissues, and compare seven methods across different gene panels and spatial sampling settings. The most useful result, in my view, is that gene-wise reconstruction accuracy and preservation of cellular expression structure do not always move together and can even change the method ranking.

Critical weaknesses

My main concern is that the benchmark construction and evaluation are hard to follow. I had to read these sections more than once to reconstruct the basic workflow. In particular, although cellular recovery is the central objective, the authors do not clearly explain how predictions with different native output resolutions are mapped to matched cellular expression profiles. The four spatial grids, three offset seeds, and cross-grid evaluation appear without enough detail on how they are built, why these particular settings were chosen, or what differs between them. I also could not easily see how gene-wise reconstruction, expression-derived clustering, annotation-based evaluation, and CRS fit together. Several terms have a specific meaning in this paper but are introduced with little background. A reader who knows ST super-resolution but not this benchmark will struggle; a worked example with input data, a prediction, and the matched cellular reference would help.

CRS is a useful idea, but its validation is not sufficient. It is tested on only one annotated breast cancer dataset with SVG100, and the correlation with annotation-based NMI uses repeated method-grid-seed observations from the same sample, so the effective evidence is narrower than the number of observations suggests. In addition, CRS excludes spots where the required correlations are undefined, including cases where a cell's predicted expression vector is constant across genes or the predicted pairwise-distance vector within a spot is constant. Those cases may be exactly where cellular heterogeneity recovery fails, but the paper does not report how often they occur or whether excluding them changes the conclusions. Beyond the single annotation-based comparison, there is no validation on other datasets or in downstream tasks, so the practical value of CRS is not yet demonstrated.

Questions for authors

Could the authors explain the complete cell-level evaluation process more clearly? In particular, how are native cell-level predictions matched to the original Xenium cells, and how are subspot and dense predictions converted into corresponding cellular expression profiles?

What are the specific differences between the four spatial grids and the three offset seeds? Why were these configurations chosen? Given that the original Xenium data already contain cell locations and expression profiles, why is it necessary to construct multiple artificial sampling grids rather than use a single fixed grid? Do these shifts correspond to realistic sampling variations? What additional information does cross-grid evaluation provide compared with evaluation on a single fixed grid?

Is computationally aggregating Xenium data into pseudo-Visium observations fully appropriate for this benchmark? Different ST platforms have substantial technical differences beyond spatial resolution, including RNA capture, detection sensitivity, and spatial background signals. Even if some of these effects may be reduced at lower resolution, I think the authors should further discuss how such platform differences could affect the benchmark results and method rankings.

The paper uses gene-wise PCC, expression-derived NMI, annotation-based NMI, and CRS to evaluate different aspects of reconstruction. Could the authors explain more clearly how these metrics differ in their evaluation targets and how they relate to the central concept of cellular recovery?

Could the authors further validate CRS across different tissues or gene panels? It would also be interesting to examine whether CRS reflects the practical usefulness of reconstructed data in downstream analyses, such as cell-cell communication inference.

Reviewer_E7ejOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Vis2Cell-Bench, a benchmark for spatial transcriptomics super-resolution. It compares representative methods using observations aggregated from Xenium datasets, with measured cellular expression serving as ground truth, and finds that method rankings vary across gene panels and spatial settings. It also shows that gene-wise accuracy does not consistently reflect the preservation of cellular structure and introduces the Cellular Recovery Score as a complementary measure of cellular recovery.

Critical strengths
  • The benchmark offers broad data coverage, encompassing 87 datasets across 13 tissue and organ categories. Four spatial partitions and three offsets yield 1,044 benchmark configurations for comparing seven methods under a common evaluation protocol.

  • The paper highlights limitations of existing methods and proposes a new evaluation metric to better assess cellular recovery.

Critical weaknesses

The relevance of this spatial transcriptomics benchmark to the broader ICLR community remains insufficiently established. While the biological motivation and dataset coverage are clear, the paper does not adequately demonstrate how the benchmark can enable meaningful advances in learning algorithms. The experiments primarily compare existing methods across gene panels, spatial settings, and evaluation metrics. These comparisons reveal performance differences but provide limited evidence that the benchmark offers actionable guidance for developing better models.

  • In particular, the finding that gene-wise accuracy does not consistently reflect cellular structure preservation is a useful diagnostic observation, but the paper does not sufficiently connect it to algorithmic insights. It does not systematically establish which architectural choices, sources of supervision, or training objectives account for this discrepancy, or demonstrate how addressing a benchmark-identified failure improves cellular recovery. Consequently, it remains unclear whether improved benchmark scores would reflect meaningful advances in learning and generalization rather than closer adaptation to the particular aggregation scheme and evaluation criteria.

  • My concern is not that a domain-specific benchmark must introduce a new algorithm or demonstrate improvements on unrelated tasks. Rather, the contribution would be stronger with a concrete demonstration of its utility for model development. The paper also provides insufficient evidence that benchmark gains translate into better generalization to real data. Without such evidence, its value is more clearly established as a specialized evaluation resource than as a contribution with broader implications for machine learning.

Questions for authors

What specific improvements in model capabilities would progress on Vis2Cell-Bench represent, and what evidence in the current study links improved benchmark performance to better cellular recovery or generalization beyond the benchmark’s controlled setting?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.