acceptodds
Under review as a conference paper at ICLR 2027

Spatial Transcriptomics Super-Resolution via Dual-Adaptive Multimodal Learning

Abstract

Conventional sequencing-based spatial transcriptomics offers broad gene coverage but aggregates signals from multiple cells, motivating histology-guided super-resolution reconstruction of finer spatial expression. Existing methods often use fixed context combinations or uniform-capacity prediction branches, limiting explicit adaptation of scale contributions and mapping capacity to local tissue and molecular information. To address this limitation, we introduce Dual-Adaptive Spatial Transcriptomics (DAST), which jointly adapts multiscale context integration and expression mapping to predict superpixel-level gene expression under spot-aggregated supervision. Specifically, DAST constructs multiscale contexts after pathology feature encoding, interacts each scale with local expression features, and dynamically learns scale contributions from the resulting multimodal representations. The resulting fused content and scale weights then jointly guide selection among experts with different hidden widths, adapting mapping capacity to the context composition. Across ten Visium HD and Xenium datasets, DAST achieves the lowest RMSE and highest SSIM compared with six other methods. Factorial comparisons and ablations provide evidence of synergistic gains from dynamic scale fusion and heterogeneous expert mapping. Downstream analyses further explore spatial domain identification and candidate tertiary lymphoid structure localization.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Spatial Transcriptomics, Super-Resolution, Multimodal Learning, Dynamic Multiscale Fusion, Mixture of Experts.

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_g58cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a super-resolution method for spatial transcriptomics. The main contribution is the Dual-Adaptive Spatial Transcriptomics which jointly adapts multiscale context integration and expression mapping to predict superpixel-level gene expression under spot-aggregated supervision. The method is evaluated on VisiumHD and Xenium datasets, showing improved performance over compared approaches.

Critical strengths
  • Overall relatively simple methods, which focused on cross-attention between modalities and multiscale contexts with dynamic fusion.

  • Results on 10 datasets show consistent improvements over existing methods.

Critical weaknesses
  • The spatial context modelling is rather simple, and the overall method is extended from HISTEX, which proposed the BCA module that is heavily used in this proposed approach. The paper shows relatively minor insights into the method design or theoretical foundations.

  • It's not clear how results of compared methods in Table 1 were obtained. The numbers do not follow the original paper either. Were the comparison fair? What experimental setup and features were used for the compared methods?

  • The use of UNI features could be very important to obtain the reported results. How would the approach work with other features?

  • It's not clear to me what was the training/testing protocol. Was the model trained and tested on each individual dataset? What was the training/testing split?

  • Visual results in Figure 3 show that the output from the proposed method is actually not very good, with lots of noise in certain regions and the boundaries between regions are noisy too. Did the improvement in numerical results mainly super-resolution or actual spatial prediction? If it's super-resolution, then if evaluation was conducted on down-sampled images, the comparative results could be quite different.

Questions for authors

Please address the weaknesses.

6 comments

Thank you for examining both the methodological contribution and the experimental basis of our work so closely. Your concerns highlight the importance of explaining what the adaptive design contributes, how the comparisons were conducted, and what the evaluation establishes. We appreciate the opportunity to address these points, as well as your recognition of the consistent improvements across ten datasets.

Below, we explain the design motivation and existing ablation evidence, the shared inputs and baseline configurations, the role of UNI features, and the section-specific training protocol. To address your observations about Figure 3, we have added regional enlargements of the original spatial-domain comparison and a separate whole-section comparison after aggregating DAST’s labels to the spot level. We will include both additions in the revised manuscript.


1. Methodological contribution beyond existing cross-modal interaction and its design rationale (W1)

We understand your concern about what DAST contributes beyond the cross-modal interaction used in previous methods. Building on prior work, including the BCA module introduced by HISTEX, DAST connects location-dependent context composition with location-dependent expression mapping.

The biological motivation comes from the role of the tissue microenvironment in gene expression: local cellular composition and surrounding tissue organization can provide relevant information over different spatial ranges. DAST lets each histological scale interact with the same local expression context and learns how much each scale contributes at each location.

The next design question is how to map this varying context to expression. DAST uses both the fused content and its scale weights to select among experts with different hidden widths. The scale weights therefore serve two connected purposes: they organize the context and inform the subsequent mapping choice. Under spot-aggregated supervision, where one observation can correspond to multiple within-spot expression distributions, these coupled adaptations provide learnable structural constraints for reconstruction.

The existing comparisons in Table 2 examine this connection using four combinations of context integration and expression mapping, with approximately matched total decoder parameter counts and the same training and evaluation protocol. On HBC_S1R1:

SettingRMSE ↓SSIM ↑
Single-scale + dense mapping0.03690.7183
Single-scale + heterogeneous experts0.03670.7196
Dynamic multiscale + dense mapping0.03780.7176
Dynamic multiscale + heterogeneous experts0.03550.7215

On this dataset, multiscale context alone does not improve the dense mapping, whereas combining it with heterogeneous experts gives the best results. Across ten datasets, replacing dense mapping with experts yields larger gains under multiscale representations on 9/10 datasets for RMSE and 8/10 for SSIM.

Table 3 further shows that the full model outperforms equal-weight fusion, homogeneous experts, and the variant without a shared expert. Permuting fusion weights or content-routing logits, or forcing any single expert, also reduces reconstruction quality. These comparisons support the contribution of jointly adapting the context and the mapping that uses it.

2. Sources of the baseline results and fairness of the experimental comparison (W2)

We appreciate why differences from the published scores raise questions about the comparison conditions. Table 1 reports results obtained by running the authors’ public implementations on the datasets evaluated here, following the training methods and configurations in their papers and code. The scores correspond to these tissue sections and target gene sets, rather than being copied from the original publications.

For HISTEX and scstGCN, we retain UNI features, 500 training epochs, and median ensembling over five model states. HISTEX’s official example specifies --epochs 500 --n-states 5; scstGCN uses the same defaults, and both prediction implementations take the median across states. DAST follows the same training duration and ensembling strategy.

For each dataset, DAST, HISTEX, and scstGCN directly reuse the same pre-extracted UNI feature file, together with the same low-resolution expression, spatial coordinates, and target genes. UNI is already used by these baselines, so sharing the file preserves their feature source while ensuring identical histological inputs. Each baseline retains its publicly released architecture and training configuration; the GPU hardware differs from that used in the original work.

Evaluation follows a common procedure: predictions and references are aligned by gene and location, independently min–max normalized per gene, and assessed using tissue-masked RMSE and whole-map SSIM with non-tissue locations set to zero, followed by aggregation across genes. HISTEX’s public evaluation scripts use this normalization and masking procedure.

Thus, the reported comparisons use common data and evaluation conditions, while preserving the methods’ original training procedures.


3. Contribution of UNI features and applicability to other histological representations (W3)

UNI is an important component of the pipeline, and your question helps distinguish its contribution from that of the downstream model. DAST, HISTEX, and scstGCN share the exact same UNI feature files across all datasets, so their performance differences are measured under identical histological representations. Tables 2 and 3 likewise hold the histological features and expression inputs fixed when evaluating the proposed components.

The pipeline arranges frozen UNI tokens at their tissue locations to form a spatial feature map. DAST’s subsequent modules operate on spatially aligned feature tensors; other encoders could be accommodated by adapting the input dimensions and spatial sampling to their feature dimensions and token spacing, followed by retraining. A controlled comparison would supply the same alternative features to DAST and the main baselines.

We have not completed that experiment, which requires new feature extraction and model training within the limited time and computational resources available during rebuttal. The current evidence supports the contribution of DAST’s downstream design under shared UNI features; performance with other encoders remains an empirical question.


4. Section-specific training, supervision, and the training/testing protocol (W4)

The key distinction is that DAST performs reconstruction within each section: a separate model uses that section’s low-resolution expression and H&E image, and its fine-grained predictions are evaluated against the high-resolution reference. Section 4.1 introduces this setting.

We construct an 8 μm reference grid from Visium HD or Xenium data and aggregate expression within circular spots to obtain low-resolution observations. During training, superpixel predictions are averaged within each spot and compared with its normalized observed expression. Individual high-resolution superpixel labels are not training targets. After training, whole-section predictions are evaluated against the reference using RMSE and SSIM.

All valid low-resolution spots are observations available for reconstructing the section, so there is no separate spatial train/validation/test split. The ten datasets are trained and evaluated independently. This protocol assesses recovery of fine-grained expression from the observed spot aggregates, rather than prediction on held-out sections.

As specified in Appendix D.2, DAST and its trained controls use 500 epochs and five independent initializations with seeds 0–4. We retain each run’s final checkpoint and take the median prediction at every location and gene; checkpoint selection does not use high-resolution validation metrics. High-resolution data are used for benchmark construction, evaluation, and Visium HD highly variable gene selection, while model optimization uses spot-level supervision.

5. Spatial-domain quality and the effect of comparing results at a common resolution (W5)

Your observations about local noise and boundaries raise a useful question about how the spatial-domain results should be compared. We have added two complementary views: regional enlargements of the original Figure 3 comparison, and a separate comparison at a common spot resolution.

First, Table 1 evaluates spatial gene-expression reconstruction. Predictions from all methods are aligned to the same high-resolution reference grid and evaluated after gene-wise normalization. RMSE measures expression errors at corresponding tissue locations, while SSIM measures local spatial structural similarity. The improvements therefore indicate closer agreement with measured reference expression on a common grid, beyond an increase in output density.

Figure 3 examines how the learned fine-grained representations support spatial-domain analysis. DAST clusters its pre-decoder multimodal representations, whereas STAGATE and GraphST identify domains from the original spot data. The colors denote cluster labels, so the local noise and boundaries concern their spatial distribution, rather than the continuous expression values evaluated in Table 1.

To examine local structures, we enlarged the hippocampal region in the original spatial-domain comparison and displayed it alongside the histological and anatomical references from the Allen Brain Atlas. These enlargements retain the original display resolution of each method.

Separately, we aggregated DAST’s superpixel cluster labels by majority voting within each spot and compared the resulting labels at exactly the same 2,896 spot locations used by STAGATE and GraphST. This whole-section comparison aligns the sampling locations and display resolution, allowing the spatial-domain patterns to be examined at a common scale.

The spot-level comparison concerns the cluster labels in Figure 3. Jointly downsampling predicted and reference expression would instead assess expression agreement at a coarser scale; Table 1 evaluates recovery at the measured high-resolution reference scale.

We will include both the regional enlargements of the original Figure 3 comparison and the separate whole-section comparison after aggregation to the spot level in the revised manuscript. We appreciate your suggestion, which provides a more direct basis for examining local structure and the effect of display resolution on the spatial-domain comparison.

Reviewer_YRRWOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces DAST, a model that embeds H&E patches at several scales, which are combined to predict expression at a given location. The novelty is adaptively learning which feature scales are important at each spot via bidirectional cross-attention modules . They apply this across ten spatial transcriptomics datasets, and find state-of-the-art performance vs existing methods, and perform ablations to quantify the impacts of different modelling components.

Critical strengths

Large set of benchmark datasets provides confidence re generalizability of results.

The idea of adaptively learning relevant length scales is I believe novel and makes sense.

Model is state of the art in the comparisons and the ablation shows all modelling choices are generally required which is cool to see.

Critical weaknesses

Table 1 needs some form of error bar (possibly over different regions within the tissue and/or genes) to assess whether the numbers are really meaningfully better. I think this is the single thing the authors could add that would maximally strengthen the paper.

The router combines fused content and scale weights to compute expert probabilities and selects the most probable expert using arg max, a nonsmooth function, which may introduce instability when used in their architecture. Could they justify this choice rather than a smooth trainable set of coefficients over experts?

In the TLS section (be sure to actually define what a TLS Is), there is a lot of controversy around what actually constitutes a TLS, and many in the field use it as shorthand for any lymphoid aggregate when strictly it's not. I think the authors can address this by adding heatmaps for the genes we expect to be expressed in a TLS individually. So first: where do the 5 genes they pick come from? Fridman 2022? I would add in some T cell markers (CD3D, CD3E, CD8A also). Can the authors then plot these individually, and show they're specifically predicting a B cell germinal center surrounded by T cells, rather than just an elevated general immune score?

What are the five visium and five xenium datasets introduced? I see these in the appendix, but please introduce in the main text, or at least reference in the appendix.

The authors used AI to polish the english but this made it painful to read in places, e.g. "Training compares mean predictions within each circular spot footprint" -> what is a "circular spot footprint"? (not really critical)

Questions for authors

I'm not sure I understand the input feature map: the authors write "Comprising UNI features and aligned RGB and coordinate channels" in 3.1 then Fig 1 seems to show "UNI + RGB + xy" -> what is the actual input representation? Are they really inputting a representation of xy co-ordinates which are not meaningful across different images?

The authors write x (the expression to be predicted) is the min-max normalized expression, but several papers in this field advocate for no scaling (e.g. predict log counts directly). How do the results change in that setting?

4 comments

We are grateful for your constructive review and your encouraging assessment of the adaptive scale design, the results across datasets, and the ablation studies. Your suggestions on performance variability and individual TLS-related markers were especially useful, prompting analyses that provide a fuller picture of the quantitative improvements and the biological patterns.

We have added gene-level summary statistics and boxplots, and retrained the model with the complete candidate marker panel to compare original and SR expression through individual B/T-associated gene maps and dual-channel overlays. We will include these analyses in the revised manuscript, together with a clearer introduction to the datasets and more direct descriptions of model operations. Below, we also explain the trainability and computational rationale of Top-1 routing, the role of coordinate inputs, and the normalization protocol, and address your question about log-count prediction.


1. Variability across genes and the interpretation of average performance improvements (W1)

We agree with your emphasis on examining performance beyond the averages in Table 1. We have added analyses on HPC (Xenium, 377 genes) and MBHD (Visium HD, 1,000 genes), reporting gene-level means and standard deviations in the table below and showing the distributions through boxplots. The boxplots display medians, interquartile ranges, whiskers, and outlying observations, helping assess whether the average improvements are also reflected in typical gene-level performance.

Within each dataset, DAST, HISTEX, and scstGCN are matched by gene name and evaluated on the same valid gene set, retaining all valid outliers. For HISTEX on HPC, the original result files underlying Table 1 were no longer available, so we reran the method using exactly the same data and configuration. The additional statistics below come from this rerun, accounting for the difference from the original Table 1 values.

DatasetMethodGenesRMSE, mean ± SD ↓SSIM, mean ± SD ↑
HPCDAST3770.030026 ± 0.0134560.790755 ± 0.129322
HPCHISTEX3770.039133 ± 0.0192110.735326 ± 0.164567
HPCscstGCN3770.036213 ± 0.0156880.753440 ± 0.134808
MBHDDAST10000.032704 ± 0.0196430.868951 ± 0.103266
MBHDHISTEX10000.048753 ± 0.0305460.810871 ± 0.140732
MBHDscstGCN10000.041162 ± 0.0496630.852769 ± 0.108068

DAST achieves the lowest mean RMSE and highest mean SSIM on both datasets, and the boxplot medians show the same advantage. The improvements therefore appear in the centers of the gene-level distributions, rather than being driven solely by a few extreme values. The tabulated standard deviations are also smaller for DAST than for both baselines on both metrics.

Here, SD is the sample standard deviation across genes, calculated from the five-model ensemble predictions used in our evaluation. It describes variation across genes, rather than training-run variability or a confidence interval for the mean.

We will include these summary statistics and boxplots in the revised manuscript, with the statistical unit clearly identified. This addition complements the average scores by showing how performance varies across genes and makes the reported improvements easier to assess.

2. Trainability and stability of Top-1 routing versus a smooth expert combination (W2)

The key implementation detail is that the selected expert retains its original softmax probability as an output weight, allowing the task loss to train the router through a continuous gate. This is described in Appendix D.1.

Let uu denote the projected fused representation and α\alpha the scale weights. Omitting spatial indices, routing and expert mapping are

a=Wuu+Wαα+bgate,π=softmax⁡(a/τ),a = W_u u + W_\alpha \alpha + b_{\mathrm{gate}}, \qquad \pi = \operatorname{softmax}(a/\tau),

k=arg⁡max⁡eπe,m=u+πkfk(u)+fsh(u).k = \arg\max_e \pi_e, \qquad m = u + \pi_k f_k(u) + f_{\mathrm{sh}}(u).

The selected probability πk\pi_k is not renormalized to one. Let LL denote the task loss and gg its gradient with respect to the output:

g=∂L∂m.g = \frac{\partial L}{\partial m}.

For a fixed selected expert, the gradient through the gate is

∂L∂aj=g⊤fk(u)πkτ(δjk−πj).\frac{\partial L}{\partial a_j} = g^\top f_k(u)\frac{\pi_k}{\tau}(\delta_{jk}-\pi_j).

Here, δjk\delta_{jk} equals 1 when j=kj=k and 0 otherwise. Thus, the continuous gate provides a gradient path to all routing logits without differentiating the discrete index. If the largest logit exceeds the second-largest by δ>0\delta>0, any perturbation satisfying ∥Δa∥∞<δ/2\|\Delta a\|_\infty<\delta/2 preserves the selected expert. This explains local trainability between switching boundaries. The balancing loss discourages concentrated assignments, while the shared expert and residual connection provide common paths independent of the selected expert.

Top-1 routing connects the fused content and scale composition to experts with different hidden widths. The content-routing permutation and forced-expert experiments in Table 3 show that disrupting the learned assignments reduces reconstruction quality, supporting location-dependent selection.

A smooth combination would instead use

msoft=u+∑e=1Eπefe(u)+fsh(u).m_{\mathrm{soft}} = u + \sum_{e=1}^{E}\pi_e f_e(u) + f_{\mathrm{sh}}(u).

This generally requires evaluating every expert. Excluding computations shared by both designs, its per-location expert cost is ∑eCe\sum_e C_e, compared with CkC_k for Top-1. Top-1 offers multiple candidate mapping capacities while evaluating only the selected routed expert, an important consideration for expression prediction over large spatial grids.

3. TLS definition, marker selection, and evidence for B/T-cell spatial organization (W3)

Your suggestion to examine individual markers helps distinguish the molecular composition of the candidate regions from their overall immune enrichment. We have extended the composite-score analysis to whole-section comparisons of original and SR expression for individual B/T-associated genes, together with B/T channel maps and overlays.

TLS are ectopic lymphoid structures that form in non-lymphoid tissues, including tumors and sites of persistent inflammation. Their organization varies with maturity; mature structures can contain B-cell follicles with germinal centers and adjacent T-cell regions.

Our candidate marker panel follows Zhang et al. (2024, iStar). The original 1,000-gene output contained five panel genes—CD74, CD79A, CD52, PTGDS, and TRBC2—which formed the initial composite score. Fridman et al. (2022) provides the biological background rather than the source of this gene combination.

Following your suggestion, we retained the original 1,000 genes, added the 17 missing panel genes, and retrained the model with 1,017 outputs, covering all 22 candidate markers. The additional single-gene figure focuses on B-cell-associated CD79A/CD79B and T-cell-associated CD3D/CD3E, showing the original spot-level expression and SR predictions across the whole tissue, with the two immune-infiltrated regions outlined.

We also constructed B/T dual-channel maps by separately averaging the normalized B- and T-associated marker pairs. For both the original and SR expression, the figure presents H&E, the B-associated channel, the T-associated channel, and their overlay. Magenta represents the B-associated signal, green the T-associated signal, and white indicates locations where both channels are high.

In the SR maps, the upper, band-shaped annotated region shows a more prominent T-associated signal. The lower region contains both B- and T-associated signals, including locations where both channels are high. These comparisons show spatial differences in lineage-associated expression within the candidate regions and provide a more specific interpretation of the composite enrichment.

CD79A/CD79B indicate B-cell-associated expression but are not specific germinal-center markers, so these patterns do not establish a B-cell germinal center surrounded by T cells. The manuscript describes the findings as “TLS-associated spatial patterns” and “candidate TLS-associated enrichment”; the additional maps examine the molecular composition and spatial relationships within those regions.

We will include these single-gene and dual-channel comparisons, the TLS definition, the complete marker panel and its source, and the visualization procedures in the revision. Your suggestion has strengthened the analysis by making it possible to inspect the B/T-associated signals underlying the composite score and compare their spatial distributions before and after reconstruction.


4. Introduction of the benchmark datasets and their analysis roles (W4)

We will introduce the datasets in the main text and directly reference Appendix Table B.1. The five Visium HD datasets cover human breast cancer (HBCHD), human ovarian cancer (HOHD), mouse brain (MBHD), mouse embryo (MEHD), and mouse kidney (MKHD). The five Xenium datasets comprise three human breast cancer samples (HBC_S1R1, HBC_S1R2, and HBC_S2), human heart (HMTC), and human pancreas (HPC).

We will also specify that HER2ST H_1 is used for the TLS-associated analysis and is outside the ten-dataset reconstruction benchmark.


5. Clarity of the language describing model operations (W5)

We agree that “circular spot footprint” can be expressed more directly. It refers to the circular tissue area covered by each spot. We will replace the sentence with:

“During training, we average the superpixel predictions within each spot and compare this average with the normalized observed spot expression using mean squared error.”

This wording describes the aggregation and loss calculation in Equation (7). We will also review the manuscript for unclear phrasing, particularly in the descriptions of inputs, context construction, and training, so that the language corresponds more directly to the model operations.

6. Composition of the input representation and the meaning of spatial coordinates (Q1)

As specified in Appendix C.1, each base-grid location is represented by 1,024-dimensional UNI features, three mean RGB values, and two normalized coordinates, concatenated into a 1,029-dimensional vector:

Fi=[fiUNI  ∥  ciRGB  ∥  pixy]∈R1029.\mathbf{F}_i = [\mathbf{f}_i^{\mathrm{UNI}} \;\Vert\; \mathbf{c}_i^{\mathrm{RGB}} \;\Vert\; \mathbf{p}_i^{xy}] \in \mathbb{R}^{1029}.

These components describe the region’s morphology, mean color, and grid center’s normalized position within the current section, respectively. Spatial alignment establishes correspondence among the inputs, while the xy channels explicitly provide position to subsequent feature transformations. Locations with similar appearances can therefore receive different positional information.

DAST is trained independently on each section, so xy provides a within-section positional reference; identical coordinates across images are not assumed to represent the same anatomy.

Coordinate features also serve a different purpose from neighborhood selection: spatial distances determine which observations enter the local expression context, whereas the concatenated xy channels provide positional information during feature processing.


7. Rationale for min–max normalization and comparison with log-count prediction (Q2)

Our evaluation follows an established practice in related spatial transcriptomics super-resolution studies, including iStar, scstGCN, and HISTEX: normalizing predicted and reference expression maps before assessing reconstruction. We apply the same normalization and metric calculations to all compared methods to support a fair and consistent comparison of spatial expression patterns.

Training and evaluation normalization serve different purposes. During training, gene-wise min–max normalization across spots places supervision targets on comparable scales, reducing the influence of expression-magnitude differences on the multi-gene mean squared error loss. During evaluation, each gene’s predicted and reference maps are independently normalized to [0,1] before computing RMSE and SSIM. Table 1 therefore evaluates normalized spatial patterns, including the locations and relative variation of expression.

Directly predicting log counts is a useful alternative. Log transformation compresses high values, while min–max normalization places genes within a common range. Under spot-aggregated supervision, a log-count objective also requires specifying the prediction scale and transformation order, because taking a logarithm after aggregation differs from aggregating log-transformed values.

Assessing this alternative requires retraining with the corresponding supervision target; transforming existing predictions would not provide that comparison. Given the time and computational resources available during rebuttal, we have not completed this experiment and cannot yet report its effect on performance. We appreciate this suggestion as a useful direction for evaluating the method under alternative expression targets.

Reviewer_CELtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a spatial transcriptomics super-resolution framework (DAST) that combines H&E and ST. DAST constructs multiscale contexts from frozen UNI features and fuses them with local expression features. The resulting scale weights organize the fused context and, together with its content, guide the selection of experts with different hidden widths, thereby mapping the expressions to fit the context composition. DAST predicts expression on an 8 μm grid. Experiments on five Visium HD and five Xenium datasets report improvements over six other methods in RMSE and SSIM. The paper also includes component ablations, routing analyses, and downstream analyses.

Critical strengths

The proposed multi-scale fusion method is interesting. It dynamically adjusts according to morphological information at different scales and shared local molecular information. This allows the contributions of different scales to vary with the local tissue context.

The experiments cover two ST platforms and include ablation studies and downstream analyses (spatial domains and TLS-associated expression).

Critical weaknesses

The authors did not explain why these three specific scales were chosen or what distinct information each contributes, nor did they directly analyze these scale-specific contributions.

The ST data were all computationally aggregated into 55 μm spots (Visium level) and used as input. However, different ST technologies vary greatly, with significant differences in molecular detection rate, RNA capture and diffusion effect, and the final sparsity. Simple calculation methods cannot reproduce these platform characteristics. Therefore, the reported similarity metrics do not establish performance on real Visium data. The downstream validation lacks independent validation of the reconstructed fine-scale molecular patterns, and no reliable advantages can be discovered from the visualization results.

Questions for authors

What motivated the choice of the 56, 112 and 224 micrometer scales and the 8 micrometer output? Although a single scale ablation study was provided, can the authors investigate the specific contributions of different scales?

The quantitative experiments are entirely dependent on the pseudo-Visium input data generated by aggregating Visium HD and Xenium expression. The issue has already been mentioned under Critical Weaknesses, so can the research results be effectively transferred to real low-resolution data?

SSIM is calculated on masked expression maps with shared zero-value backgrounds, which may affect the reported scores. Can the authors also provide SSIM specifically for the tissues and compare the results with the simple interpolation of the original low-resolution expressions?

In Figure 3, DAST generates finer spatial-domain patterns, but the overall improvement in tissue domain identification is not always clearly visible. Can the authors provide independent anatomical annotations or quantitative indicators of the accuracy of regions and boundaries?

In Figure 4, additional smoothing and background suppression were performed on the super-resolution expressions. Can the authors explain this difference and provide the results under a consistent visualization process? Additionally, can measured high-resolution expression data or independently validated TLS annotations be provided as references?

5 comments

Thank you for your thoughtful review and the specific suggestions on our evaluation and biological analyses. We appreciate your recognition of the adaptive multiscale design and the experiments across two platforms. Your comments have been particularly helpful in examining how the reconstruction results are evaluated and how the downstream findings are supported.

We have considered each concern and carried out additional analyses, including tissue-interior SSIM evaluation, a linear interpolation baseline, clustering stability analysis, and individual-gene visualizations with consistent processing. We have also added regional enlargements for the spatial-domain comparison. Below, we explain the relevant design choices and evidence, and indicate the visual comparisons we will include in the revised manuscript.


1. Rationale for the spatial scales and their individual contributions (W1; Q1)

The biological motivation comes from the role of the tissue microenvironment in local gene expression: cellular composition, neighboring cells, and surrounding tissue organization can all provide relevant information at different spatial ranges. DAST therefore combines local morphological detail with broader tissue context and learns their relative contributions from both histological and expression information.

The specific scales connect this motivation to the measurement geometry and feature sampling. As described in Section 3.2 and Appendices B.2 and C.1, images are calibrated to 0.5 μm/pixel, so a 16 × 16-pixel token corresponds to an 8 × 8 μm² region. This defines the output grid and matches the high-resolution reference grid. A 7 × 7-token window spans 56 × 56 μm², with a side length close to the 55 μm input spot diameter. Nonoverlapping average pooling with kernel sizes and strides of 2 and 4 increases token spacing to 16 and 32 μm, producing 112 × 112 and 224 × 224 μm² windows. The three branches thus cover progressively broader tissue neighborhoods while each retaining 49 tokens, with predictions remaining on the 8 μm grid. All datasets use the same candidate scales, and their fusion weights vary by location.

The ablations in Table 3 help explain their contributions. The 56 μm branch performs best among the single-scale variants, while the full model outperforms all three, supporting the importance of local detail and the complementary value of broader context. Equal-weight fusion performs worse than learned fusion and does not improve upon the 56 μm-only setting, indicating that the benefit depends on how the scales are combined.

The fusion-weight permutation experiment provides a test: it exchanges complete scale-weight vectors across locations while preserving local features and the scale order within each vector. The resulting performance decrease supports matching scale contributions to the tissue content at each location. Appendix Table E.1 shows the same advantage of the full model over single-scale and equal-weight settings on HOHD, from the other platform.

Together, these existing results support using the local window as a foundation, incorporating surrounding tissue information, and adapting the contribution of each scale to the local context.


2. Applicability of pseudo-Visium results to real low-resolution data (W2; Q2)

You raise an important distinction between spatial aggregation and platform-specific measurement processes. The pseudo-Visium benchmarks enable quantitative evaluation against measured high-resolution references. As described in Appendix B.2, we aggregate Visium HD or Xenium expression within 55 μm circular regions arranged in a Visium-like staggered layout. Training uses the aggregated spot expression, while evaluation tests recovery of the underlying fine-grained expression.

For real low-resolution data, DAST learns from the target section’s own expression and H&E image, following the section-specific training protocol in Section 4.1. Application therefore does not require transferring parameters trained on another platform. Figure 4 provides an application to real low-resolution HER2ST H_1 data, showing spatial correspondence between reconstructed TLS-associated expression and annotated immune-infiltrated regions.

Spatial aggregation captures the mixing of fine-grained signals within spots, but does not reproduce platform-specific capture efficiency, RNA diffusion, or background noise. Section-specific training uses the target sample’s actual observations, although these measurement factors can still affect reconstruction. Direct validation of fine-scale accuracy on real Visium data requires matched high-resolution measurements.

The two analyses thus provide complementary evidence: the pseudo-Visium benchmarks evaluate reconstruction against high-resolution references, while HER2ST demonstrates application to real low-resolution data. We will clarify this scope and the platform measurement differences in the revised manuscript.

3. Influence of shared zero backgrounds on SSIM and comparison with interpolation (Q3)

This suggestion prompted additional tissue-interior SSIM evaluation and a linear interpolation baseline on MBHD and HBC_S1R1, covering 1,000 and 313 genes, respectively.

The additional evaluation averages SSIM only over local windows lying entirely within valid tissue. It excludes background and boundary-crossing windows while retaining genuine zero-expression locations within tissue. Linear interpolation uses only low-resolution spot expression and spatial coordinates, without histology. All methods share the output grid, evaluation genes, and normalization rules. The decrease below is whole-image SSIM minus tissue-interior SSIM.

DatasetMethodRMSE ↓Whole-image SSIM ↑Tissue-interior SSIM ↑SSIM decrease
MBHDLinear interpolation0.1635440.5027160.3127160.190000
MBHDHISTEX0.0487530.8108710.6276440.183227
MBHDscstGCN0.0411620.8527690.7091970.143572
MBHDDAST0.0327040.8689510.7612410.107710
HBC_S1R1Linear interpolation0.1898250.4115090.3847220.026787
HBC_S1R1HISTEX0.0498280.6401290.6271610.012968
HBC_S1R1scstGCN0.0451290.6659340.6463820.019552
HBC_S1R1DAST0.0355310.7215340.7092980.012236

DAST retains the highest SSIM after excluding the shared background and has the smallest decrease on both datasets. Its advantage over scstGCN increases from 0.016182 to 0.052044 on MBHD and from 0.055600 to 0.062916 on HBC_S1R1. The advantage therefore persists under strictly tissue-interior evaluation.

DAST also outperforms linear interpolation: tissue-interior SSIM is 0.761241 versus 0.312716 on MBHD and 0.709298 versus 0.384722 on HBC_S1R1. Together with lower RMSE, these results support improved recovery of normalized spatial expression patterns beyond simple interpolation.

These additional analyses support that DAST’s reconstruction advantage persists when shared background regions are excluded and extends beyond simple interpolation of spot-level expression


4. Independent anatomical evidence for spatial-domain and boundary accuracy (W2; Q4)

To make the local structures easier to assess, we have added enlarged views of the hippocampal region for DAST, STAGATE, and GraphST, alongside histological and anatomical references from the Allen Brain Atlas. The whole-section views identify the selected regions, and the enlargements allow closer inspection of local layered structures, regional shapes, and boundary contours.

As described in Appendix F.1, the atlas serves as a qualitative anatomical reference. Differences in section planes and individuals mean it cannot provide point-by-point ground-truth labels for this section, for which matched independent region and boundary annotations are currently unavailable.

We also examined whether the displayed partitions depend on clustering initialization. With the input representations and cluster count fixed, we repeated K-means using 10 random seeds and computed all 45 pairwise adjusted Rand indices (ARIs) over the same 456,551 valid locations. Before small-region cleanup, mean ARI was 0.9930 ± 0.0034, with a minimum of 0.9862; after cleanup, mean ARI was 0.9926 ± 0.0033. These results show high reproducibility across initializations.

The enlarged comparisons provide anatomical context, while the ARI analysis measures clustering reproducibility rather than anatomical accuracy. We will include the regional enlargements in the revised manuscript, with a clear explanation of the atlas reference.

5. Consistency of visualization and independent reference evidence for TLS-associated patterns (W2; Q5)

We agree that consistent visualization makes the comparison more informative. In the original Figure 4, the low-resolution composite signal is displayed at measured spot locations, while SR predictions undergo smoothing and background suppression to highlight candidate enriched regions. These operations affect visualization only and do not participate in training, as described in Appendix F.2.

We have added consistently processed comparisons of the original and SR expression maps for CD52, CD74, CD79A, and PTGDS, alongside the manually annotated whole-section H&E image. These four genes belong to the five-gene panel used for the original composite score; the additional figure examines their individual spatial distributions.

In these maps, CD52, CD74, and PTGDS show elevated signals in both outlined regions, while CD79A enrichment is more prominent in the lower region. The original and SR maps allow the spatial patterns of each marker to be compared directly, while the H&E annotations provide a tissue reference for interpreting their locations and shapes.

For this section, matched measured high-resolution expression and independently validated TLS annotations are currently unavailable. The yellow outlines identify manually annotated immune-infiltrated regions. The manuscript accordingly describes “TLS-associated spatial patterns” and “candidate TLS-associated enrichment,” focusing on their spatial organization and correspondence with these regions rather than claiming confirmed TLS detection.

We will include these additional single-gene comparisons in the revised manuscript, together with the visualization procedures and annotation details. We appreciate your suggestions, which have helped us examine the molecular signals underlying the composite score more closely.

Comments on the paper

2 comments
Area_Chair_3qW2OpenReview ↗

Please check authors' rebuttal and participate in the discussion

Dear reviewer,

The discussion phase is coming to an end. Please read the authors’ rebuttal and post a brief follow-up in the discussion thread. Let us know which of your concerns the response addresses and which remain open.

Please post any follow-up questions in the thread visible to the author as soon as possible, so that the author has time to respond and provide clarification. I also encourage you to engage with one another’s assessments—especially where your perspectives differ—and to note whether any counterarguments or discussions lead you to revise your own assessment.

Thank you for your effort and thoughtful review. Thank you again!

AC

AuthorsOpenReview ↗

General Response

For clarity, we refer to Reviewers CELt, YRRW, and g58c as R1, R2, and R3, respectively.

We are sincerely grateful to all three reviewers for the care they took in reading our paper. We were encouraged by R2’s recognition of the novelty of learning which tissue scales matter at each location and by their positive assessment of the ten-dataset benchmark and ablation studies. We also appreciated hearing from R1 that the revisions addressed most of their concerns, and from R3 that our response clarified the experimental setup and comparisons.

We have addressed each comment in our point-by-point responses and revised the manuscript in light of the discussion. Below, we summarize DAST’s core contributions, the additional experiments we conducted, and the clarifications provided in the manuscript and reviewer responses.


Core Contributions of Our Work

DAST jointly learns how to combine tissue context across scales at each superpixel and how to map that context to gene expression. The learned scale weights guide both fusion and expert selection.

  1. Post-encoder dynamic multiscale multimodal fusion. DAST builds three tissue contexts from a shared histology feature map. At each superpixel, interactions between histology and local spot expression determine how much each scale contributes.
  2. Fusion-conditioned heterogeneous expert mapping. The fused representation and its scale weights guide the choice among experts with different hidden widths. In Table 2, replacing a dense decoder with heterogeneous experts yielded a larger RMSE reduction on 9 of 10 datasets and a larger SSIM improvement on 8 of 10 datasets with dynamic multiscale fusion than with single-scale input.
  3. Cross-platform evaluation and downstream analysis. Against six baselines (Table 1), the full DAST model trained with spot-level supervision achieved both the lowest RMSE and the highest SSIM on all ten Xenium and Visium HD datasets. We used the learned representations to identify spatial domains and the reconstructed expression maps to examine tertiary lymphoid structure (TLS) markers.

Experimental Updates During the Rebuttal

  • Alternative histology encoder (R3, follow-up responses). Whether we used UNI or HIPT features, DAST achieved higher SSIM and lower RMSE than both HISTEX and scstGCN on HBC_S1R1 (Xenium) and MBHD (Visium HD). These results show that DAST’s adaptive context integration and expression mapping retain their advantage over baselines when UNI is replaced by HIPT.
  • Reconstruction within tissue (R1, reviewer response). We added a linear interpolation baseline and restricted SSIM to windows entirely within tissue. On HBC_S1R1 and MBHD, DAST had the highest tissue-interior SSIM of the four methods and the smallest decrease from whole-image SSIM.
  • Gene-level performance (R2, Appendix B.6). We report boxplots, means, and standard deviations across 377 matched genes on HPC and 1,000 on MBHD. Of the three methods, DAST had the lowest mean and median RMSE and the highest mean and median SSIM on both datasets.
  • Spatial-domain comparisons (R1, R3). Figure 3 adds hippocampal enlargements from the original comparison alongside the Allen Brain Atlas reference (R1). Appendix F.1 separately compares whole-section maps at the same spot locations after aggregating DAST’s superpixel labels within each spot (R3).
  • Immune marker patterns (R1, R2). Figure 4 and Appendix F clarify how measured and predicted expression maps were displayed (R1). New individual-gene maps and B- and T-cell marker overlays show which markers contribute to each annotated region and where their signals are adjacent or overlap (R2).

Clarifications in the Manuscript and Reviewer Responses

  • Training and evaluation. The main text introduces the ten datasets and explains how predictions are averaged within each spot for supervision (R2). Our response to R3 details section-specific training, how baseline results were obtained, and how high-resolution measurements were used for evaluation.
  • Tissue scales and adaptive design. We explained each scale’s physical coverage and how its contribution was assessed (R1). The factorial comparisons in Table 2 clarify how adaptive fusion complements expert mapping (R3).
  • Input representation and Top-1 routing. We explained how coordinates encode position within a section and how retaining the selected expert’s routing probability lets the reconstruction loss train the router (R2).

The revisions are highlighted in blue for ease of reference. Several of the analyses above grew directly out of questions raised during the review, and we are grateful to the reviewers for sharing those perspectives with us. To the Area Chair and all three reviewers, please accept our warm thanks for the time and thought you have given our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.