acceptodds
Under review as a conference paper at ICLR 2027

SPARSE: Semantic-Preserving Adapter for Robust Spatial Embeddings

Abstract

Spatial transcriptomic representations should incorporate local tissue context without erasing expression-derived cell-type structure. We introduce SPARSE, a parameter-efficient hierarchical adapter for frozen transcriptomic embeddings. Within each centered spatial neighborhood, SPARSE builds a multi-resolution token hierarchy through feature-guided token merging with coordinate propagation, spatial rotary position encoding, cross-resolution attention, and top-down fusion. Masked expression reconstruction encourages retention of molecular information. With scGPT as the main backbone, SPARSE improves spatial-domain identification on two held-out human-brain slices and the held-out final stage of a mouse-embryo dataset. Across four transcriptomic backbones, it retains or improves cell-type separability on a held-out mouse-heart slice. On a held-out mouse whole-brain section, the same fixed embeddings support cell-type label transfer, neighborhood-composition prediction, and cell-wise tissue-region classification. Controlled comparisons show gains beyond parameter-matched adapters and graph-Laplacian regularization. With approximately 5M trainable adapter parameters, SPARSE complements dedicated spatial pretraining.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Spatial Transcriptomics, Representation Learning, Transcriptomic Foundation Models, Parameter-Efficient Adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hjXwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper proposes SPARSE, a comparably lightweight spatial adapter on frozen transcriptomic foundation-model embeddings that combines TCFormer-style token merging, 2D spatial RoPE, cross-resolution attention, and FPN-style fusion, trained with masked expression reconstruction. SPARSE applied to spatial domain identification tasks.

Critical strengths
  • The paper address timely and important research question, cell representation learning from spatial transcriptome.
Critical weaknesses
  • Novelty: The contribution is largely combinatorial (TCFormer-style token merging + 2D RoPE + FPN-style fusion + masked reconstruction) rather than an original solution to a known challenge or a new framing of the problem, and it is unclear which component addresses anything specific to spatial transcriptomics.
  • Clarity: The term “hierarchical” is overstated. A hierarchy requires that coarser levels structurally constrain finer ones, but neither SPBlock nor HSF enforces this: cross-scale attention is dense across all fine tokens regardless of cluster membership, and HSF explicitly avoids assignment-based upsampling. The method should be toned down to “multi-scale.”
  • Significance
    • Baselines. Standard spatial-domain methods mentioned in related work (STAGATE, GraphST, BayesSpace) are not compared.
    • Comparing methods by NMI/ARI measures how well each embedding’s clusters overlap with ground-truth labels, so how the ground truth was defined is critical. Yet the labels driving the main results are vague: the human-brain “17 domain types” (Table B3) are never described (cortical layers? anatomical regions?). The human-lung analysis (Fig. E1) uses only section-level healthy-vs-cancer labels, not cell-level ground truth. It’s unclear whether ground truth labels are reliable or appropriate for this task.
    • It is unclear whether the gains come from better embeddings or from Leiden behaving differently on a 64-dim embedding than on the higher-dimensional competitors.
Questions for authors
  • Line 1430, authors stated that “The ablation study is the stated exception: it uses seed 42 and averages the two held-out human-brain regions.” Why is it an exception? Every other table follows the three-seed mean ± std rule, but the ablation (Table 2 / E1) was run once with seed 42 and averaged over the two held-out brain regions.
6 comments
AuthorsOpenReview ↗

Initial Response to Reviewer hjXw(Part 1/2)

Thank you for the constructive feedback. We agree that the manuscript should state more clearly what SPARSE contributes, what its multiresolution structure does and does not enforce, and what conclusions the evaluations support. We address each point below:

W1: Novelty

Just as FPN and ViTDet build multiresolution representations to contextualise local image features, spatial transcriptomics benefits from interpreting a cell’s expression profile in the context of the surrounding tissue. The core novelty of SPARSE is to apply this principle to spatial transcriptomics by decoupling gene-expression encoding from multiresolution spatial modelling: rather than retraining the transcriptomic backbone, SPARSE builds local tissue context at multiple resolutions from existing cell embeddings and integrates that context back into the target cell’s representation. To realise this idea, SPARSE draws on TCFormer’s feature-guided token merging. Physical coordinates determine which neighbours are included, while token features determine how those neighbours are grouped. The representation thus becomes coarser across stages without expanding the spatial neighbourhood. Unlike TCFormer, SPARSE uses the same weights to aggregate token features and physical coordinates during merging, then uses the resulting coordinates in the encoder’s cross-resolution attention via 2D RoPE. During fusion, TCFormer’s original MTA upsamples coarse features according to cluster assignments before further fusion. SPARSE’s Hierarchical Stage Fusion (HSF) instead lets finer tokens directly attend to the fused tokens from the next coarser stage, without restricting attention to their original merge groups. The final fused representation of the centre cell is the adapter’s output.

W2: Use of “hierarchical”

We agree that “multiresolution” is more precise than “hierarchical” when describing the overall adapter. Token merging creates fine-to-coarse assignments, but SPBlock does not restrict attention to tokens assigned to the same cluster, and HSF does not use assignment-based upsampling. In the camera-ready version, we will use “multiresolution” to describe different token granularities within the same neighborhood. For ease of reference, we retain HSF in this response.

W3(a): Additional spatial-domain baselines

The table below reports the three-seed mean NMI and ARI for STAGATE and GraphST on the two held-out human-brain regions and the held-out mouse-embryo stage. In the camera-ready version, we will add these results to the main comparison table and include visualizations of the corresponding spatial-domain assignments.

MethodHuman Brain Region 1 NMI % ↑Human Brain Region 1 ARI % ↑Human Brain Region 2 NMI % ↑Human Brain Region 2 ARI % ↑Mouse Embryo E16.5 NMI % ↑Mouse Embryo E16.5 ARI % ↑
STAGATE52.0142.3835.2521.0758.0141.93
GraphST56.4241.9726.5916.0064.0046.58
SPARSE57.5347.0338.9224.5065.7749.81

We did not include BayesSpace in this embedding-based comparison. Its native spatial clustering returns labels rather than a learned spatial embedding that can be passed to the same 15-NN graph construction and Leiden pipeline. Its native spatial adjacency also assumes a regular lattice, whereas our evaluation data contain irregularly positioned cells. BayesSpace labels could be scored directly with NMI and ARI, but that would be a comparison of different clustering pipelines rather than of embeddings under a shared readout.

W3(b): Human-brain annotations and the purpose of the lung analysis

The human-brain reference labels are not exclusively cortical layers or exclusively anatomical regions. They include cortical developmental zones such as the cortical plate, subplate, and ventricular zone, as well as anatomical structures such as the striatum and thalamus. Region 1 and Region 2 contain 17 and 16 distinct values in the reference annotation field, respectively, including the retained NA category (Table B3, Appendix B.3). We will replace “domain types” with “reference annotation categories” and clarify that NMI and ARI measure agreement with these annotations. The two holdouts are defined by coordinates; we do not claim that they are independent physical sections.

The lung analysis asks whether SPARSE embeddings reflect disease-associated differences in cellular states, as shown in Figure E1 (Appendix E.1). It is exploratory: healthy and cancer labels are assigned at the section level, not to individual cells as verified malignant or non-malignant labels. The observed separation may also reflect differences in cell-type composition, section identity, or technical factors. We will clarify that this visualization is not evidence of cell-level malignancy classification or of disease-state discrimination within a matched cell type.

AuthorsOpenReview ↗

Initial Response to Reviewer hjXw(Part 2/2)

W3(c): Sensitivity to output dimensionality

We tested SPARSE output dimensions of 128, 256, and 512 on both human brain and mouse embryo. The encoder and the 64-dimensional HSF fusion layers were kept fixed in width; only the terminal projection, its output normalization, and the corresponding reconstruction-head input changed. We trained each higher-dimensional variant with three seeds and used the same downstream clustering procedure. The 64-dimensional results are the original manuscript references and were not rerun.

The table below shows spatial-domain identification results for SPARSE at output dimensions of 64, 128, 256, and 512. We have added these results to Table E14 (Appendix E.8).

Output dimensionHuman brain Region 1 NMI % ↑Human brain Region 1 ARI % ↑Human brain Region 2 NMI % ↑Human brain Region 2 ARI % ↑Mouse embryo E16.5 NMI % ↑Mouse embryo E16.5 ARI % ↑
6457.53 ± 0.5847.03 ± 0.6638.92 ± 3.0824.50 ± 4.6265.77 ± 0.9749.81 ± 1.19
12855.17 ± 1.9142.98 ± 5.1739.92 ± 2.9825.90 ± 5.0164.92 ± 1.1551.04 ± 2.45
25652.75 ± 1.1742.76 ± 1.3036.26 ± 1.8322.87 ± 4.0165.35 ± 0.5851.46 ± 1.30
51251.27 ± 3.4336.91 ± 3.8337.14 ± 2.4024.89 ± 3.5066.04 ± 0.7050.58 ± 1.76

When averaged across the two human-brain regions, the 128-dimensional results are close to the 64-dimensional reference, while the 256- and 512-dimensional variants have lower means. On mouse embryo, NMI remains similar across dimensions, and mean ARI does not decrease at the higher dimensions. These results show that SPARSE can perform well beyond 64 dimensions, but its sensitivity to output dimensionality varies by dataset. Lower-dimensional embeddings do not necessarily yield better clustering results. Because the higher-dimensional variants were retrained, this experiment does not isolate the effect of dimensionality on the 15-NN graph or Leiden, nor does it replace a dimension-matched comparison across methods.

Q1: Three-seed ablation results

Tables 2 and E1 originally reported two-region averages for seed 42 only. We have now added seeds 0 and 123 for all five ablations.

The table below summarizes the human-brain component ablation results across three seeds. We averaged the two regions within each seed before computing the mean and sample standard deviation. We have updated Table 2 (Section 4.4) with these results.

Model variantTwo-region mean NMI % ↑Two-region mean ARI % ↑
w/o spatial neighborhood38.34 ± 1.5128.43 ± 4.12
w/o HSF43.29 ± 0.2934.43 ± 0.75
w/o rCTM45.43 ± 1.2733.57 ± 1.22
w/o cross-scale attention45.31 ± 1.1633.93 ± 1.34
w/o token score45.73 ± 2.2032.79 ± 1.00
Full scGPT+SPARSE48.23 ± 1.7835.76 ± 2.62

Table E1 (Appendix E.3) also reports the results for each region separately. The full configuration has the highest mean NMI and ARI when averaged over the two regions.

Thank you for the detailed response and additional experiments. The inclusion of STAGATE and GraphST, the three-seed ablations, and the clarification of “multiresolution” terminology address several of my concerns. I also appreciate the more careful interpretation of the lung visualization. However, my main concerns about novelty and the strength of the empirical evidence remain.

  1. Novelty. The response clarifies the architecture and its differences from TCFormer, particularly the aggregation of physical coordinates and the assignment-free fusion. However, it does not sufficiently establish why these adaptations constitute a substantive methodological contribution specific to spatial transcriptomics. The ablations support the utility of individual components within SPARSE, but do not establish the advantage of the proposed design over simpler ways of incorporating spatial context into existing cell embeddings.
  2. Dimensionality and the clustering comparison. The new experiments demonstrate that SPARSE can perform reasonably across several output dimensions, but they do not resolve the original concern about comparisons across methods. As the authors acknowledge, retraining variants with different terminal projections does not isolate the effect of embedding dimensionality on graph construction and Leiden clustering. Moreover, the fusion representation remains 64-dimensional. A comparison using a common dimensionality across methods would more directly address this concern.

Overall, the rebuttal improves the clarity and completeness of the evaluation. I therefore increase my Presentation score from 2 to 3. However, my central concerns about methodological novelty and the interpretation of the reported gains remain, so I maintain my overall score of 2 (Weak rejection).

Reviewer_DtNVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a roughly 5M-parameter hierarchical adapter for frozen transcriptomic embeddings. It combines spatial neighborhoods, feature-based token merging with coordinate propagation, spatial RoPE, cross-resolution attention, and top-down fusion, trained with masked expression reconstruction. Evaluation covers spatial-domain identification, cell-type separability, and several readouts of a fixed representation. The main results are promising, particularly the held-out embryo-stage result and the human-brain region-1 ARI improvement.

Critical strengths
  1. Adapting a frozen expression backbone is a useful practical formulation. The architecture explicitly combines molecular features and spatial context, and the cell-level output can support multiple downstream tasks.
  2. Table 1 reports the best mean NMI and ARI on all three evaluations. Region-1 ARI is 47.03 versus 39.48 for the strongest alternative mean; embryo-stage ARI is 49.81 versus 45.37. Table 3 evaluates semantic retention across four backbones.
  3. The reporting is unusually explicit about coordinate-defined brain holdouts, within-section probes, fixed epoch-30 checkpoints, three-seed variability, and limits of statistical inference. Appendix D.4 also reports adaptation cost and matched reconstruction objectives.
Critical weaknesses
  1. The neighborhood target in Appendix C.3 (p. 22) uses softmax of cell-type counts rather than count-normalized proportions. This can emphasize the dominant class and does not directly measure composition recovery. Please verify the implementation and compare normalized-count targets.
  2. Generalization and semantic-retention evidence remain limited by coordinate-defined brain holdouts, single held-out slices, and within-section probe splits. Three-seed variation does not capture biological replication, while weighted F1 may conceal rare-class losses.
  3. The Linear and Laplacian controls cannot aggregate neighboring molecular features to reconstruct masked tokens (D.4). Adding frozen scGPT and simple neighborhood aggregation would better isolate the hierarchy’s contribution.
Questions for authors
  1. Does the composition advantage persist with count-normalized targets?
  2. How does SPARSE compare with frozen scGPT and simple neighborhood aggregation under the same splits?
  3. Can you report per-class F1 changes and sensitivity to a spatially blocked probe split?
7 comments
AuthorsOpenReview ↗

Initial Response to Reviewer DtNV(Part 1/3)

Thank you for the constructive feedback. We address the neighborhood target, the comparison with simple aggregation, and the sensitivity to spatial separation below.

W1 & Q1: Niche prediction with count-normalized targets

As stated in Appendix C.3, the original implementation applied softmax directly to neighborhood cell-type counts. We agree that this changes the target and that the original MAE should not be interpreted as an error in predicting cell-type proportions.

We repeated the evaluation using the original within-section splits of the held-out mouse whole-brain section. The revised target divides each cell-type count by the actual number of neighbors. Softmax is retained only at the probe output to ensure that the predicted proportions sum to one. We kept the embeddings and data splits fixed and evaluated all methods at the same three probe learning rates.

The table below reports the full results, we have added these results to Table E10 (Appendix E.7).

MethodNiche MAE % , LR 0.05 ↓Niche MAE % , LR 0.01 ↓Niche MAE % , LR 0.005 ↓
PCA6.171 ± 0.1103.573 ± 0.0233.615 ± 0.019
Novae1.653 ± 0.0102.266 ± 0.0112.870 ± 0.018
Nicheformer6.007 ± 0.4713.405 ± 0.5332.430 ± 0.040
scGPT-spatial2.212 ± 0.0442.408 ± 0.0092.536 ± 0.005
scGPT + Linear5.722 ± 0.0232.276 ± 0.0572.163 ± 0.030
scGPT + Transformer5.436 ± 0.2591.995 ± 0.1582.017 ± 0.206
scGPT + Laplacian Reg5.917 ± 0.6652.178 ± 0.0942.167 ± 0.036
scGPT + SPARSE1.643 ± 0.0031.709 ± 0.0471.830 ± 0.056

SPARSE retained the lowest mean MAE among the methods shown at all three learning rates.

W3 & Q2: Comparison with frozen scGPT and simple neighborhood aggregation

We agree that the original Linear and Laplacian Reg controls do not directly aggregate neighboring molecular features in their forward pass. To test whether simple aggregation is sufficient, we added two comparisons for human-brain spatial-domain identification. Frozen scGPT uses each cell’s original embedding. Frozen scGPT + Mean200 uniformly averages the embeddings of the center cell and its 199 nearest spatial neighbors, using the same neighborhoods as SPARSE and no trainable parameters. All three methods use the same coordinate-defined held-out regions and clustering protocol: a 15-nearest-neighbor graph and Leiden resolution 0.8.

The table below reports the results for both held-out regions, we have added these results to Table E13 (Appendix E.8).

MethodHumain brain Region 1 NMI % ↑Humain brain Region 1 ARI % ↑Humain brain Region 2 NMI % ↑Humain brain Region 2 ARI % ↑
Frozen scGPT40.20 ± 1.0128.53 ± 1.6928.49 ± 0.4216.79 ± 1.26
Frozen scGPT + Mean20044.29 ± 0.297.68 ± 0.1734.87 ± 0.124.56 ± 0.05
Frozen scGPT + SPARSE57.53 ± 0.5847.03 ± 0.6638.92 ± 3.0824.50 ± 4.62

SPARSE achieved higher mean NMI and ARI than both controls in both regions. Thus, aggregation alone did not reproduce the gains of the full adapter.

AuthorsOpenReview ↗

Initial Response to Reviewer DtNV(Part 2/3)

W2 & Q3: Semantic retention and sensitivity to spatial separation

We used labeled cells from mouse-heart section P14_1_2 for feature standardization and as the reference for distance-weighted 15-nearest-neighbor label transfer to all 23,786 cells in held-out section P14_1_1. The target section was excluded from adapter training. Its labels were used only to score predictions and define the boundary subset, which contains 16,962 cells with at least 10% of their 199 nearest spatial neighbors annotated as a different cell type.

The table below shows cross-section cell-type label transfer from P14_1_2 to all cells of held-out P14_1_1. We added these results to Table E8 (Appendix E.6).

BackboneModelAccuracy % ↑Weighted F1 % ↑Macro F1 % ↑
scGPTFrozen78.1274.0729.66
+Linear74.78 ± 4.3070.63 ± 4.2227.01 ± 2.67
+Transformer68.16 ± 4.6161.80 ± 5.8113.20 ± 4.15
+Laplacian Reg72.32 ± 2.3366.15 ± 1.2014.09 ± 1.03
+SPARSE80.53 ± 1.7476.46 ± 2.1631.24 ± 1.22
scFoundationFrozen83.9379.5132.61
+Linear82.25 ± 2.2777.85 ± 2.1231.78 ± 0.60
+Transformer66.65 ± 5.2357.92 ± 8.4911.37 ± 5.83
+Laplacian Reg71.64 ± 3.0064.93 ± 1.4011.97 ± 2.74
+SPARSE82.76 ± 1.0277.93 ± 0.8931.48 ± 0.78
scBERTFrozen52.0250.418.03
+Linear71.58 ± 2.2563.67 ± 2.399.98 ± 0.89
+Transformer63.90 ± 1.5259.22 ± 3.8810.23 ± 2.23
+Laplacian Reg74.83 ± 3.1167.26 ± 2.3110.95 ± 0.58
+SPARSE69.10 ± 2.2664.53 ± 2.4912.48 ± 1.06
GeneformerFrozen83.6678.1930.67
+Linear80.66 ± 1.7574.89 ± 1.8828.94 ± 1.24
+Transformer77.66 ± 8.4870.83 ± 10.8023.99 ± 8.86
+Laplacian Reg74.78 ± 2.6168.08 ± 1.9716.22 ± 1.04
+SPARSE84.16 ± 0.8579.62 ± 1.5333.60 ± 1.85

And this table shows results for boundary cells, updated in Table E9 (Appendix E.6).

BackboneModelAccuracy % ↑Weighted F1 % ↑Macro F1 % ↑
scGPTFrozen72.7368.0825.02
+Linear72.21 ± 3.5466.63 ± 3.1722.92 ± 2.83
+Transformer67.43 ± 2.5860.21 ± 2.5912.00 ± 3.64
+Laplacian Reg68.09 ± 4.0360.43 ± 1.899.16 ± 0.57
+SPARSE74.14 ± 1.7269.15 ± 1.7526.59 ± 0.98
scFoundationFrozen77.9372.0928.47
+Linear77.08 ± 1.1171.00 ± 1.0827.60 ± 0.85
+Transformer68.15 ± 2.0459.60 ± 3.2111.12 ± 4.09
+Laplacian Reg68.51 ± 4.9360.61 ± 2.419.28 ± 0.67
+SPARSE76.59 ± 0.8370.27 ± 0.9427.18 ± 0.66
scBERTFrozen52.6951.787.12
+Linear70.86 ± 1.6461.04 ± 1.149.07 ± 0.63
+Transformer65.30 ± 2.7659.12 ± 1.579.89 ± 1.55
+Laplacian Reg71.38 ± 2.9962.08 ± 1.599.65 ± 0.48
+SPARSE65.57 ± 1.4460.29 ± 1.7510.63 ± 1.67
GeneformerFrozen77.7570.5826.39
+Linear76.67 ± 1.0068.84 ± 1.2824.90 ± 1.47
+Transformer73.98 ± 3.4865.88 ± 4.9020.84 ± 6.43
+Laplacian Reg69.14 ± 3.3161.10 ± 1.739.31 ± 0.63
+SPARSE78.27 ± 0.6772.03 ± 1.6529.70 ± 1.96

Across both the full query set and the boundary subset, SPARSE achieved the highest mean macro F1 for scGPT, scBERT, and Geneformer. It also achieved the highest mean accuracy and weighted F1 for scGPT and Geneformer in both sets. For scBERT, Laplacian Reg led on weighted F1, while Frozen remained the strongest variant for scFoundation. Figures E3 and E4 show the cell-type proportions and per-cell-type F1 changes relative to Frozen for all query cells and the boundary subset, respectively. The proportions reveal a strong imbalance in cell-type frequencies. In this setting, weighted F1 is influenced more by common cell types, whereas macro F1 gives equal weight to all cell types, including those represented by very few cells. The two metrics provide complementary summaries.

AuthorsOpenReview ↗

Initial Response to Reviewer DtNV(Part 3/3)

On mouse whole-brain section 1.099, we evaluated a diagonal split with a buffer between the training and test blocks. The buffer prevented overlap across the split in both the SPARSE input neighborhoods and the neighborhoods used to construct the composition targets. We kept each method’s embeddings fixed. On this split, SPARSE achieved the lowest mean niche MAE at all three probe learning rates and the highest mean tissue-level accuracy and weighted F1, while its cell-level scores were close to those of scGPT-spatial. Results are shown below, and reported in Table E11, Appendix E.7.

MethodCell accuracy % ↑Cell weighted F1 % ↑Niche MAE % , LR 0.05 ↓Niche MAE % , LR 0.01 ↓Niche MAE % , LR 0.005 ↓Tissue accuracy % ↑Tissue weighted F1 % ↑
PCA27.59 ± 0.1019.59 ± 0.167.756 ± 0.0645.887 ± 0.0316.121 ± 0.09251.77 ± 0.3057.60 ± 0.35
Novae39.18 ± 0.0035.88 ± 0.002.476 ± 0.0093.495 ± 0.0174.485 ± 0.03183.60 ± 0.2485.62 ± 0.20
Nicheformer53.23 ± 0.0046.85 ± 0.006.258 ± 0.5073.458 ± 0.1012.727 ± 0.08021.09 ± 1.0925.54 ± 1.72
scGPT-spatial83.55 ± 0.0082.70 ± 0.002.612 ± 0.0482.756 ± 0.0152.938 ± 0.01458.42 ± 0.5665.63 ± 0.41
scGPT + Linear27.62 ± 6.9923.60 ± 3.786.732 ± 1.3293.281 ± 0.2413.199 ± 0.37640.63 ± 19.2741.21 ± 17.82
scGPT + Transformer40.69 ± 5.7237.58 ± 5.125.788 ± 0.2883.024 ± 0.5092.929 ± 0.31711.35 ± 5.5215.18 ± 7.68
scGPT + Laplacian Reg24.16 ± 2.4623.24 ± 2.505.965 ± 0.0003.083 ± 0.1023.099 ± 0.13143.06 ± 8.3642.46 ± 7.84
scGPT + SPARSE83.13 ± 0.3782.56 ± 0.302.376 ± 0.0662.472 ± 0.1292.635 ± 0.14484.68 ± 3.3686.32 ± 2.51

To test a separate question—whether probes can transfer between sections without fitting on the test section—we also fitted the downstream readouts on section 1.098 and evaluated them on all 39,026 cells in section 1.099. Section 1.099 was excluded from SPARSE adaptation, the adapter was not retrained, and test labels were used only for evaluation.

The cross-section results are reported below and updated in Table E12 (Appendix E.7).

MethodCell accuracy % ↑Cell weighted F1 % ↑Niche MAE % , LR 0.05 ↓Niche MAE % , LR 0.01 ↓Niche MAE % , LR 0.005 ↓Tissue accuracy % ↑Tissue weighted F1 % ↑
PCA17.37 ± 0.0210.42 ± 0.026.020 ± 0.0393.245 ± 0.0023.338 ± 0.05379.39 ± 0.0978.48 ± 0.07
Novae48.42 ± 0.0042.47 ± 0.001.657 ± 0.0062.123 ± 0.0062.548 ± 0.01683.26 ± 0.1282.37 ± 0.06
Nicheformer44.16 ± 0.0038.77 ± 0.006.037 ± 0.5343.612 ± 0.2792.522 ± 0.11139.30 ± 1.4637.25 ± 0.76
scGPT-spatial13.22 ± 0.003.09 ± 0.004.968 ± 0.0704.989 ± 0.0285.044 ± 0.02219.79 ± 11.048.41 ± 6.33
scGPT + Linear8.16 ± 4.374.54 ± 0.926.240 ± 1.1845.100 ± 1.4015.108 ± 0.98910.54 ± 14.093.87 ± 6.47
scGPT + Transformer17.57 ± 8.4514.32 ± 9.204.576 ± 0.6824.098 ± 0.4864.204 ± 0.43727.13 ± 1.1312.32 ± 1.05
scGPT + Laplacian Reg14.88 ± 15.028.54 ± 10.395.611 ± 0.8426.109 ± 0.4835.914 ± 0.28518.66 ± 14.167.69 ± 6.34
scGPT + SPARSE87.64 ± 1.2087.05 ± 1.271.789 ± 0.1051.787 ± 0.0891.878 ± 0.10086.48 ± 0.3185.42 ± 0.46

On this source–target pair, SPARSE had the highest mean Cell and Tissue scores among the methods shown. The Niche ranking depended on the probe learning rate: Novae had the lowest MAE at 0.05, whereas SPARSE had the lowest MAE at 0.01 and 0.005.

Reviewer_MB3yOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SPARSE is a parameter-efficient hierarchical adapter for frozen transcriptomic embeddings. It builds a multi-resolution token hierarchy within each centered spatial neighborhood through feature-guided token merging and fuses coarser context back to cell-level outputs. Masked expression reconstruction encourages retention of molecular information. With scGPT as the main backbone, SPARSE improves spatial-domain identification on held-out human-brain regions and a held-out mouse-embryo stage. Further experiments examine cell-type separability across transcriptomic backbones and the utility of fixed embeddings on a held-out mouse whole-brain section. Controlled comparisons show gains beyond parameter-matched adapters and graph-Laplacian regularization.

Critical strengths
  1. The adaptation setting is practical and well motivated. SPARSE incorporates local tissue context while keeping the transcriptomic backbone frozen. Feature-guided token merging, coordinate propagation, and Hierarchical Stage Fusion provide a coherent architecture for this purpose.
  2. The spatial-domain experiments include useful controlled comparisons. Comparisons with parameter-matched adapters use matched masking and reconstruction settings. Component ablations and Leiden-resolution sensitivity further support the contribution of the complete SPARSE configuration.
Critical weaknesses
  1. The neighborhood-composition target does not represent cell-type proportions. Appendix C.3 applies softmax to neighborhood cell-type counts. This can strongly emphasize the most abundant type and changes the target as neighborhood size increases, even when proportions remain similar. Consequently, lower MAE on this target does not directly establish better neighborhood-composition prediction, weakening the interpretation of the niche-level results.
  2. The semantic-preservation evaluation is too narrow to establish the main motivation. The mouse-heart experiment reports aggregate accuracy and weighted F1 using reference and query cells from the same held-out slice. It does not compare semantic retention with the parameter-matched adapters or examine rare cell types and boundary cells. The results therefore leave unclear whether the proposed hierarchy better retains distinctions between molecularly different neighboring cells.
  3. The evidence for robust transfer remains limited. The human-brain holdouts are coordinate-defined regions without confirmed independent physical-section identities. In the mouse whole-brain experiment, downstream probes are fitted and evaluated within the same held-out section using random cell-level splits. Although representation adaptation excludes the evaluation data, these settings provide limited evidence for generalization across independent specimens.
Questions for authors
  1. Is softmax applied directly to neighborhood cell-type counts in the implementation, as stated in Appendix C.3? If so, do the conclusions hold when the targets are cell-type proportions obtained by normalizing those counts?
  2. How does SPARSE compare with the parameter-matched adapters on the semantic-retention task? Can the authors provide cell-type-specific results and results for boundary cells to assess whether aggregate improvements conceal losses in cellular discrimination?
2 comments
AuthorsOpenReview ↗

Initial Response to Reviewer MB3y(Part 1/2)

Thank you for pointing out the distinction between the original niche target and cell-type proportions, and between holding out a section during representation adaptation and holding it out during downstream probe fitting.

W1 & Q1: Neighborhood composition prediction with count-normalized targets

As described in Appendix C.3, the original implementation applied softmax directly to neighborhood cell-type counts. We agree that the resulting MAE measures error against a transformed target and cannot be interpreted as error in predicting cell-type proportions.

We repeated the evaluation on the original within-section splits of the held-out mouse whole-brain section. For each neighborhood, we now divide each cell-type count by the actual number of neighbors. Softmax is used only on the probe outputs to produce predicted proportions. For each method, we kept its embeddings fixed. At each learning rate, all methods used the same data splits and probe settings.

The results below are also reported in Table E10 (Appendix E.7), separately from the original softmax-count evaluation described in Appendix C.3. MAE (%) is averaged over neighborhood sizes 100, 200, and 500 and reported as mean ± sample standard deviation over seeds 0, 42, and 123.

MethodNiche MAE % , LR 0.05 ↓Niche MAE % , LR 0.01 ↓Niche MAE % , LR 0.005 ↓
PCA6.171 ± 0.1103.573 ± 0.0233.615 ± 0.019
Novae1.653 ± 0.0102.266 ± 0.0112.870 ± 0.018
Nicheformer6.007 ± 0.4713.405 ± 0.5332.430 ± 0.040
scGPT-spatial2.212 ± 0.0442.408 ± 0.0092.536 ± 0.005
scGPT + Linear5.722 ± 0.0232.276 ± 0.0572.163 ± 0.030
scGPT + Transformer5.436 ± 0.2591.995 ± 0.1582.017 ± 0.206
scGPT + Laplacian Reg5.917 ± 0.6652.178 ± 0.0942.167 ± 0.036
scGPT + SPARSE1.643 ± 0.0031.709 ± 0.0471.830 ± 0.056

SPARSE retains the lowest mean MAE among the compared methods at all three learning rates under the original within-section protocol.

W2 & Q2: Semantic retention across cell types and among boundary cells

Cross-section cell-type label transfer. We used only labeled cells from mouse-heart section P14_1_2 for feature standardization and as the reference for distance-weighted 15-nearest-neighbor label transfer to all 23,786 cells in held-out section P14_1_1. The target section was excluded from adapter training. Its labels were used only to score predictions and define the boundary subset, which contains 16,962 cells with at least 10% of their 199 nearest spatial neighbors annotated as a different cell type.

The table below shows cross-section cell-type label transfer from P14_1_2 to all cells of held-out P14_1_1. We added these results to Table E8 (Appendix E.6).

BackboneMethodAccuracy % ↑Weighted F1 % ↑Macro F1 % ↑
scGPTFrozen78.1274.0729.66
+Linear74.78 ± 4.3070.63 ± 4.2227.01 ± 2.67
+Transformer68.16 ± 4.6161.80 ± 5.8113.20 ± 4.15
+Laplacian Reg72.32 ± 2.3366.15 ± 1.2014.09 ± 1.03
+SPARSE80.53 ± 1.7476.46 ± 2.1631.24 ± 1.22
scFoundationFrozen83.9379.5132.61
+Linear82.25 ± 2.2777.85 ± 2.1231.78 ± 0.60
+Transformer66.65 ± 5.2357.92 ± 8.4911.37 ± 5.83
+Laplacian Reg71.64 ± 3.0064.93 ± 1.4011.97 ± 2.74
+SPARSE82.76 ± 1.0277.93 ± 0.8931.48 ± 0.78
scBERTFrozen52.0250.418.03
+Linear71.58 ± 2.2563.67 ± 2.399.98 ± 0.89
+Transformer63.90 ± 1.5259.22 ± 3.8810.23 ± 2.23
+Laplacian Reg74.83 ± 3.1167.26 ± 2.3110.95 ± 0.58
+SPARSE69.10 ± 2.2664.53 ± 2.4912.48 ± 1.06
GeneformerFrozen83.6678.1930.67
+Linear80.66 ± 1.7574.89 ± 1.8828.94 ± 1.24
+Transformer77.66 ± 8.4870.83 ± 10.8023.99 ± 8.86
+Laplacian Reg74.78 ± 2.6168.08 ± 1.9716.22 ± 1.04
+SPARSE84.16 ± 0.8579.62 ± 1.5333.60 ± 1.85
AuthorsOpenReview ↗

Initial Response to Reviewer MB3y(Part 2/2)

This table below shows results for boundary cells, updated in Table E9 (Appendix E.6).

BackboneMethodAccuracy % ↑Weighted F1 % ↑Macro F1 % ↑
scGPTFrozen72.7368.0825.02
+Linear72.21 ± 3.5466.63 ± 3.1722.92 ± 2.83
+Transformer67.43 ± 2.5860.21 ± 2.5912.00 ± 3.64
+Laplacian Reg68.09 ± 4.0360.43 ± 1.899.16 ± 0.57
+SPARSE74.14 ± 1.7269.15 ± 1.7526.59 ± 0.98
scFoundationFrozen77.9372.0928.47
+Linear77.08 ± 1.1171.00 ± 1.0827.60 ± 0.85
+Transformer68.15 ± 2.0459.60 ± 3.2111.12 ± 4.09
+Laplacian Reg68.51 ± 4.9360.61 ± 2.419.28 ± 0.67
+SPARSE76.59 ± 0.8370.27 ± 0.9427.18 ± 0.66
scBERTFrozen52.6951.787.12
+Linear70.86 ± 1.6461.04 ± 1.149.07 ± 0.63
+Transformer65.30 ± 2.7659.12 ± 1.579.89 ± 1.55
+Laplacian Reg71.38 ± 2.9962.08 ± 1.599.65 ± 0.48
+SPARSE65.57 ± 1.4460.29 ± 1.7510.63 ± 1.67
GeneformerFrozen77.7570.5826.39
+Linear76.67 ± 1.0068.84 ± 1.2824.90 ± 1.47
+Transformer73.98 ± 3.4865.88 ± 4.9020.84 ± 6.43
+Laplacian Reg69.14 ± 3.3161.10 ± 1.739.31 ± 0.63
+SPARSE78.27 ± 0.6772.03 ± 1.6529.70 ± 1.96

SPARSE achieved the highest mean macro F1 for scGPT, scBERT, and Geneformer on both the full query set and the boundary subset. It also achieved the highest mean accuracy and weighted F1 for scGPT and Geneformer in both sets. For scBERT, Laplacian Reg led on weighted F1, while Frozen remained the strongest variant for scFoundation.

Figures E3 and E4 (Appendix E.6) show the cell-type proportions and per-cell-type F1 changes relative to Frozen for all query cells and the boundary subset, respectively. The proportions reveal a strong imbalance in cell-type frequencies. In this setting, weighted F1 is influenced more by common cell types, whereas macro F1 gives equal weight to all cell types, including those represented by very few cells. The two metrics provide complementary summaries.

W3: Transfer evaluation with coordinate-defined holdouts and cross-section probing

For the human brain, the test regions were held out by spatial coordinates. The human-brain dataset was provided as a single stitched spatial map without original section identifiers, so we could not split the data by physical section. Instead, we used spatial coordinates to hold out two regions and adapted SPARSE only on the remaining cells Table B3 and Figure B2 (Appendix B.3).

In the original mouse whole-brain experiment, probes were fitted and evaluated within section 1.099, which was held out from SPARSE adaptation. In the cross-section evaluation, we embed both sections with the same SPARSE checkpoint, fit probes on 1.098, and use 1.099 labels only for scoring.

The cross-section results below are also reported in Table E12 (Appendix E.7). Values are mean ± sample standard deviation over seeds 0, 42, and 123, expressed as percentages. Niche MAE uses count-normalized targets and is averaged over neighborhood sizes 100, 200, and 500.

MethodCell accuracy % ↑Cell weighted F1 % ↑Niche MAE % , LR 0.05 ↓Niche MAE % , LR 0.01 ↓Niche MAE % , LR 0.005 ↓Tissue accuracy % ↑Tissue weighted F1 % ↑
PCA17.37 ± 0.0210.42 ± 0.026.020 ± 0.0393.245 ± 0.0023.338 ± 0.05379.39 ± 0.0978.48 ± 0.07
Novae48.42 ± 0.0042.47 ± 0.001.657 ± 0.0062.123 ± 0.0062.548 ± 0.01683.26 ± 0.1282.37 ± 0.06
Nicheformer44.16 ± 0.0038.77 ± 0.006.037 ± 0.5343.612 ± 0.2792.522 ± 0.11139.30 ± 1.4637.25 ± 0.76
scGPT-spatial13.22 ± 0.003.09 ± 0.004.968 ± 0.0704.989 ± 0.0285.044 ± 0.02219.79 ± 11.048.41 ± 6.33
scGPT + Linear8.16 ± 4.374.54 ± 0.926.240 ± 1.1845.100 ± 1.4015.108 ± 0.98910.54 ± 14.093.87 ± 6.47
scGPT + Transformer17.57 ± 8.4514.32 ± 9.204.576 ± 0.6824.098 ± 0.4864.204 ± 0.43727.13 ± 1.1312.32 ± 1.05
scGPT + Laplacian Reg14.88 ± 15.028.54 ± 10.395.611 ± 0.8426.109 ± 0.4835.914 ± 0.28518.66 ± 16.147.69 ± 6.34
scGPT + SPARSE87.64 ± 1.2087.05 ± 1.271.789 ± 0.1051.787 ± 0.0891.878 ± 0.10086.48 ± 0.3185.42 ± 0.46

On this split, SPARSE has the highest mean cell-level and tissue-level accuracy and weighted F1 among the methods shown. The niche ranking depends on the learning rate: Novae has the lowest MAE at 0.05, while SPARSE has the lowest MAE at 0.01 and 0.005. This experiment shows that the downstream probes can transfer without fitting on the test section.

Comments on the paper

4 comments

The rebuttal period is about to close. We hope the authors can respond to our concerns before the deadline, otherwise, I will reconsider my score.

We apologize for the delay in responding and appreciate your patience. We spent considerable time running additional experiments to address your concerns, and we have now completed them and included the results in our response below. Thank you for taking the time to review these results.

Thank you for your reply. I will review it carefully and communicate with you.

Area_Chair_V6dAOpenReview ↗

Please check authors' rebuttal and participate in the discussion

Dear reviewer,

The discussion phase is coming to an end. Please read the authors’ rebuttal and post a brief follow-up in the discussion thread. Let us know which of your concerns the response addresses and which remain open.

Please post any follow-up questions in the thread visible to the author as soon as possible, so that the author has time to respond and provide clarification. I also encourage you to engage with one another’s assessments—especially where your perspectives differ—and to note whether any counterarguments or discussions lead you to revise your own assessment.

Thank you for your effort and thoughtful review. Thank you again!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.