acceptodds
Withdrawn from ICLR 2027

MIST: Multi-Scale Spatial Context for Gene Expression Prediction from Histology

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting spatial gene expression from standard H&E images could extend molecular analysis to tissues without spatial transcriptomic measurements. A standard Transformer applies self-attention over spots but ignores where each spot sits in the tissue, whereas a spot's molecular state depends on tissue context at multiple spatial scales. Building on this insight, we introduce MIST, a Multi-context Spatial Transformer that augments the self-attention backbone with two context pathways applied in every layer: a coordinate-invariant local pathway that attends over spatial \(k\)-nearest-neighbor spots, and a slide pathway that broadcasts a whole-slide summary to every spot; the model predicts all spots in a single forward pass. Because coordinates enter only through pairwise distances, the local pathway is invariant to translations, rotations, and reflections when image features are fixed. In intra-cohort evaluation across six human organ cohorts from HEST-1k, MIST improves mean Pearson correlation over the strongest evaluated baseline by 9.7%, 11.6%, and 11.4% for 10-, 50-, and 100-gene panels, respectively, with the lowest average MSE and MAE. MIST further improves leave-one-organ-out transfer by 6.0% over the next strongest baseline, attains the best expression-derived domain recovery (highest mean ARI and NMI at every clustering resolution, and higher NMI than the closest baseline on 84% of held-out slides), and reproduces spatial expression patterns of selected cancer-marker genes. Together, these results show that adding local and slide context to a vanilla Transformer backbone is an effective approach to spatial gene expression prediction from histology.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Spatial transcriptomics, gene expression prediction, computational pathology, histology, spatial context, Transformer, spatial k-nearest neighbors, whole-slide context, H&E imaging, pathology foundation models.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_U7MXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MIST, a framework for predicting spatial gene expression from H&E histology images. The authors argue that standard self-attention does not explicitly model local spatial neighborhoods or the overall tissue state and propose adding local spatial and slide-level pathways to a Transformer backbone. The model extracts spot-level image features using frozen pathology foundation models and combines global self-attention, spatial kNN-based local attention, and an attention-pooled slide summary broadcast to all spots within each layer. The resulting representations are used to predict gene expression at each spot. The authors evaluate MIST on six human organ cohorts from HEST-1k under intra-cohort, pooled multi-organ, and leave-one-organ-out settings, reporting improvements over the selected baselines. They further assess the model through component ablations, spatial structure recovery, and cancer-marker gene prediction analyses.

Critical strengths

Evaluation across multiple generalization settings. The experiments cover intra-cohort prediction, pooled multi-organ training, and leave-one-organ-out transfer, along with different gene panels and image encoders.

Consistent gains over the evaluated baselines. MIST reports improved average performance across multiple settings and gene panels. Under the default 50-gene HVG setting, it outperforms the evaluated baselines on all six cohorts. Reporting PCC alongside MSE and MAE provides complementary measures of correlation and prediction error.

Evaluation beyond aggregate prediction metrics. Spatial-domain recovery and cancer-marker gene analyses provide additional evidence about the structure and biological patterns captured by the predictions.

Availability of code and implementation details. The authors provide an anonymous code repository and describe the main architectural and training hyperparameters, improving transparency and providing a basis for independent inspection and reproduction.

Critical weaknesses
  1. Limited methodological novelty. Local–global context modeling is well established in histology-based gene expression prediction. The paper does not sufficiently explain what limitation of existing approaches is addressed by combining global self-attention, local kNN attention, and a pooled slide summary in every layer.

  2. Incomplete empirical verification of invariance. The invariance propositions largely follow from the architectural construction. However, Table 4 reports changes in PCC, which is insensitive to positive scaling and additive shifts. This does not directly verify prediction-level invariance; prediction differences should be reported after aligning spot order where necessary.

  3. Missing closely related baselines. TRIPLEX integrates target, neighborhood, and whole-slide information, while STFlow incorporates local spatial attention and coordinate-invariant modeling. Omitting these methods leaves the advantage of MIST’s context-modeling strategy insufficiently evaluated.

  4. Limited evidence for the slide-summary module. Adding the slide summary alone does not consistently improve performance. Its gains over global-plus-local attention are small (+0.009/+0.003/+0.005 PCC), without a separately reported significance test. Moreover, the SKCM and LUNG ablations use UNI rather than the main UNI+CONCH configuration.

  5. Inconsistent results and dataset descriptions. Results differ between Tables 1 and 8 under reportedly matched settings. In Table 8, the UNI+CONCH cohort values average to 0.344 for Hist2ST and 0.319 for STEM, versus the reported 0.335 and 0.310. Appendix D also describes all samples as Visium, although the corresponding benchmark includes Xenium data. The numerical discrepancies and any spatial aggregation should be clarified.

Minor weakness. Figure 1 does not clearly illustrate summary broadcasting, residual fusion, or coordinate versus feature flows. Its caption also refers to panels (a) and (b), but these labels are absent from the figure.

Questions for authors

see weakness

Reviewer_swtSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MIST, a spatial Transformer for predicting gene expression from H&E images. It incorporates local spatial-neighbor attention and whole-slide context into standard self-attention. Experiments on six HEST-1k organ cohorts demonstrate improved prediction accuracy, cross-organ generalization, and spatial structure recovery compared with existing methods.

Critical strengths
  1. The proposed method is simple and intuitive, effectively incorporating local spatial neighborhoods and whole-slide context into a standard Transformer.
  2. The experiments cover six organ cohorts with multiple gene panels and cross-organ evaluation, providing relatively comprehensive evidence of the method's effectiveness.
  3. The ablation studies and spatial structure recovery experiments help validate the contributions of different components beyond standard prediction accuracy.
Critical weaknesses
  1. The methodological novelty appears somewhat limited, as combining local spatial relationships with global tissue context has already been explored in Hist2ST, STFlow, and HEXST. Moreover, MIST uses coordinates only for kNN construction without explicitly modeling relative distances or directions, leaving the effectiveness of its spatial representation insufficiently justified.
  2. The necessity of the slide-level pathway remains unclear, since global self-attention already captures interactions across all spots, and the slide pathway alone does not improve performance. Furthermore, the advantages of learning slide context through attention pooling over incorporating pretrained whole-slide representations, such as TITAN and CARE, are not established.
Questions for authors
  1. Given that local and global spatial modeling has already been explored in Hist2ST, STFlow, and HEXST, what is the key methodological distinction of MIST beyond combining existing attention mechanisms?
  2. Why are spatial coordinates used only for kNN construction rather than explicitly incorporated into attention through relative positional encoding or distance-based Fourier features?
  3. Since global self-attention already aggregates information across all spots, what additional information does the slide-level pathway provide, especially considering that it does not improve performance when used alone?
  4. Pretrained whole-slide foundation models such as TITAN and CARE can provide tissue-level representations. What advantages does the proposed attention-pooled slide context offer over incorporating these pretrained features?
Reviewer_ra3POpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

MIST predicts spot-level gene expression from frozen histology embeddings and spatial coordinates. Each Transformer layer combines global self-attention with spatial-neighbor attention and an attention-pooled slide summary. The paper evaluates prediction across organs, gene panels, and encoders, including held-out-organ transfer, and examines expression-derived domain recovery and cancer-marker patterns.

Critical strengths
  1. Multi-context spatial modeling. MIST augments a self-attention Transformer with local spatial-neighbor attention and an attention-pooled slide summary in every layer. The local pathway is invariant to coordinate-frame transformations under the stated fixed-feature assumptions.

  2. Evaluation across multiple prediction settings. The paper reports improvements across organs, gene panels, and image encoders, including held-out-organ transfer. Component ablations and spatial-versus-random neighbor comparisons examine the contributions of the added context pathways.

Critical weaknesses
  1. The methodological advance is limited. Spatial-neighbor attention and attention pooling are established operations, and the main design combines them with standard self-attention. The coordinate-invariance property follows directly from distance-based graph construction. The paper does not sufficiently establish what new modeling capability or generalizable insight this combination provides beyond existing spatial-context approaches.

  2. The need for the slide-context branch is unclear. Global self-attention already accesses all spots, so the broadcast summary overlaps with an existing information pathway. Slide context alone does not consistently improve performance, and its additional benefit on top of local context is modest. A simpler pooling control would help establish whether the specific slide pathway is necessary.

  3. The experiments do not fully isolate the proposed architecture's value. The random-neighbor control supports spatial locality, but does not distinguish MIST from simpler spatial attention or smoothing. A focused comparison under the same objective and comparable capacity is needed. Baseline adaptations and loss configurations should also be specified to separate architectural gains from optimization differences.

Questions for authors
  1. What benefit does MIST provide over a simpler spatial-context model under matched training objectives and capacity?
  2. Does the slide pathway offer a reproducible advantage over a simpler shared pooling summary?
Reviewer_MxDuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

MIST predicts spatial gene expression from histology using frozen pathology embeddings and a Transformer combining global self-attention, local spatial kNN attention, and pooled slide-level context. The authors evaluate it across six HEST-1k cohorts and report improvements over ST-Net, Hist2ST, BLEEP, and STEM, including in leave-one-organ-out generalization.

Critical strengths

• Clear experimental evaluation. The authors provide controlled ablations, including comparisons between spatial and random neighborhoods, supporting the benefit of incorporating local spatial information. • Generalization analysis. Leave-one-organ-out experiments and spatial-domain recovery analyses go beyond standard within-cohort prediction accuracy.

Critical weaknesses
  1. Limited architectural novelty. The combination of global self-attention, local spatial attention, and slide-level pooling is largely incremental. TRIPLEX (2024), MISO (2025), and HiST (2026) already explore closely related approaches to integrating local and global tissue context. The proposed coordinate-frame invariance is also a straightforward consequence of constructing neighborhoods using Euclidean distances, rather than a new architectural contribution.
  2. Missing comparisons against relevant recent methods. The evaluation compares MIST against ST-Net, Hist2ST, BLEEP, and STEM, but omits several closely related methods, including TRIPLEX, MISO, MERGE, and HiST. Given the architectural overlap, these comparisons are essential for establishing whether MIST represents a meaningful advance.
  3. Limited evaluation scope. Most experiments predict only 10–100 selected genes, and some cohorts contain just two or three independent slides. Patient-level independence is not established. Furthermore, spatial-domain recovery remains modest in absolute terms (ARI 0.091, NMI 0.110), limiting the strength of the biological conclusions.
Questions for authors
  1. What is the specific architectural novelty relative to TRIPLEX, MISO, and HiST?
  2. Can the authors provide comparisons against these and other recent methods under matched experimental settings?
  3. The ablations suggest that local spatial attention accounts for most of the improvement. Is the full three-pathway architecture necessary compared with simpler, parameter-matched alternatives?
  4. Do the improvements persist when predicting substantially larger gene panels?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.