acceptodds
Withdrawn from ICLR 2027

Depth-Resolved Gene Expression Estimation from Tissue Morphology and Sparse Spatial Transcriptomics

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

10 views

Abstract

Most spatial transcriptomics assays measure gene expression on a limited number of two-dimensional tissue sections, leaving molecular variation across depth largely unobserved. We investigate whether volumetric tissue imaging can provide the structural context needed to estimate this missing molecular information from sparse 2D measurements. We introduce VISTA (Volumetric Inference from Sparse Transcriptomic Anchors), a structure-guided framework for estimating depth-resolved gene-expression fields from 3D tissue imaging using 2D spatial transcriptomics measurements as molecular anchors. VISTA combines a frozen pathology foundation encoder with multi-scale structural encoding, depth-aware attention, and 3D graph propagation to integrate local and long-range tissue context, while a Schr\"odinger-bridge objective along depth encourages coherent predictions that remain consistent with observed ST measurements and preserve 3D structural organization. A single formulation covers three settings that differ in how directly depth-resolved prediction can be checked. On registered serial sections, predictions are checked against held-out measured sections, under supervision as thin as a single revealed plane. On a serial volume in which every section is sequenced, depth accuracy is measured directly against held-out expression and reported against a split-half noise ceiling that bounds what is recoverable; VISTA attains roughly half of it, against a few percent for interpolation. On continuous volumes, where only one plane is sequenced and no method can report per-depth accuracy, we isolate the contribution of depth with a control that replaces the volume by its own projection. Across settings, VISTA improves agreement with measured ST and produces more coherent and depth-sensitive molecular estimates than existing baselines. These results support structure-guided volumetric modeling as a practical approach for extending sparse sectional transcriptomic measurements into depth-resolved molecular representations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
3D gene expression, volumetric tissue imaging, structure-guided modeling, multimodal learning, spatial transcriptomics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DU4VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VISTA predicts gene expression across tissue depth from morphology and a small number of spatial transcriptomics measurements. It combines a frozen UNI2-h encoder with multi-scale features, depth attention, graph propagation, and a bridge-matching loss. The paper evaluates the model on HER2+ serial sections, Open-ST, and two holotomography (HT) samples. The strongest result is on Open-ST: with the two outer sections revealed and 17 interior sections withheld, VISTA achieves a depth correlation of 0.4223, compared with 0.2713 for TRIPLEX.

Critical strengths

VISTA outperforms all reported baselines in depth correlation, gene correlation, and MSE (Table 4). The depth-correlation gap over TRIPLEX, 0.4223 versus 0.2713, is the clearest reason to take the method seriously. This result carries more weight than the visual smoothness of a predicted volume.

The Schrödinger-bridge loss between revealed planes is novel in this context and conceptually stronger than heuristic smoothness priors, enabling non-monotone axial variation while remaining anchored to endpoints.

Use of a noise ceiling on Open-ST and explicit discussion of why per-site correlation can be misleading for depth are commendable.

Critical weaknesses

Appendix A.8 defines the loss between two revealed planes and averages over locations strictly between them (Eqs. 8-9). In the single-section HER2+ protocol and the HT experiments, there is no second measured endpoint. Under the written definition, the set P is empty. Does the implementation turn the loss off, construct a second endpoint, or use another loss? The paper does not say, yet repeatedly states that the objective is unchanged across settings.

The paper shows that the full model predicts better, but does not establish which depth components produce the gain.

The component ablations are on HT, where depth-resolved expression is not measured (Table 9). Smoothness and alignment with morphology cannot, by themselves, establish that gene expression is correct at a given depth.

Questions for authors
  1. What is the exact training objective when only one section is revealed? Please specify how the empty-P case is handled, where any second endpoint comes from, how xi,zx_{i,z} is obtained, and how physical depth intervals enter the loss.

  2. What physical tissue does the HT pooling operator represent?

Section 3.8 says the measurement integrates the entire depth column, while Section 4.3 describes a single measured plane. Please give the HT volume thickness, axial spacing, and the position and thickness of the Visium section. Why should expression pooled over all 65 HT slices match that measurement? If the measurement covers only part of the volume, does restricting supervision to that support change the results?

  1. How exactly is depth correlation computed and averaged over genes and trajectories, including centering, missing depths, and constant predictions?
Reviewer_Qk8ROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

VISTA predicts depth-resolved gene expression from tissue images and sparse spatial transcriptomic data. It employs a frozen pathology encoder, multiscale 2D/3D features, depth attention, graph propagation and a bridge-matching regulariser. In the HT setting, predictions across depth are pooled and compared with the measured Visium expression during training. It is assessed on HER2+ serial sections, Open-ST and two gastric cancer HT samples, including within-sample and cross-sample prediction.

Critical strengths
  • The paper looks at a very useful problem indeed: predicting 3D gene expression from tissue images and extremely limited RNA. It could potentially go a long way to reducing the amount of experimental work that would otherwise be required.

  • The method reports improvements over the compared methods under limited RNA supervision, and there is further evidence of generalisation across patients and samples.

Critical weaknesses
  • The bridge loss is not clear in the single-section case. Appendix A.8, Eqs. (8)–(9), requires two revealed planes, but in a number of the experiments we only have one. The endpoints and intermediate expression states used in those cases are not sufficiently specified.

  • The HT supervision and validation require much stronger justification. Section 3.8 takes the matched Visium measurement as an aggregate over the entire depth column without any clear explanation as to what the acquisition geometry is that would support that assumption. The HT experiments are only on two samples, and there are no molecular measurements at the hidden depths, so spatial coherence and morphology agreement cannot by themselves establish the accuracy of expression predictions at hidden depths.

  • The contribution of the proposed components is not well isolated. Table 9 only reports ablations on HT rather than Open-ST, where we can directly inspect the hidden-depth expression. The projection control also removes the 3D convolution and depth attention, altering both the input and the architecture.

  • Baseline adaptation and coverage still need to be validated. ASIGN performs considerably worse than in the original paper under different evaluation regimes. There is no controlled reproduction to tell us how much of the performance drop is due to the change in task versus baseline adaptation. STFlow has public code but is not evaluated. Section 4.3 refers to Appendix G for the reasons for excluding VORTEX, but Appendix G provides no VORTEX-specific explanation.

Questions for authors
  • How is the bridge loss worked out with just one measured section? How do you obtain the endpoints and intermediate states without using withheld expression labels?

  • What tissue thickness and acquisition geometry would be sufficient to justify pooling the entire HT depth column against a single Visium measurement?

  • On Open-ST, what happens to performance without the bridge loss or with simpler depth regularisation? Can the projection comparison maintain the architecture?

  • Can you reproduce ASIGN’s original results and determine which protocol changes contribute to the deterioration in performance? For Table 2, which sections, if any, of test samples D and H had expression revealed to each method, and how was ASIGN’s measured RNA input used?

  • Why is STFlow not included in the serial-section experiments, and what specific input or supervision requirement prevents applying VORTEX?

Reviewer_6jQ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VISTA to estimate depth-resolved 3D gene expression fields from volumetric tissue imaging and sparse 2D spatial transcriptomics anchors. The model combines a frozen pathology encoder with multi-scale 2D and 3D token branches, axial-depth attention, graph propagation, and a depth-regularization term, evaluating predictions across stacked serial sections, a fully sequenced Open-ST volume, and continuous holotomography volumes paired with a single Visium plane. While evaluating against held-out Open-ST depths relative to a split-half noise ceiling is a sensible validation strategy, the mathematical definition of the depth bridge objective breaks down in single-plane settings and the continuous-volume claims rely on indirect proxies.

Critical strengths
  1. Benchmarking depth-resolved predictions on the fully sequenced Open-ST volume and normalizing depth correlation against a split-half noise ceiling provides a concrete way to separate genuine depth variation from in-plane correlation.
  2. The paper notes a useful failure mode of standard metrics in 3D settings: simple interpolation across revealed faces can achieve high 2D site correlation by matching gene-wise means (while recovering almost none of the actual expression variation along depth).
Critical weaknesses
  1. The Schrodinger-bridge objective is overstated and inconsistent with the single-plane experiments. In the appendix, this loss involves no stochastic process or distribution transport, but simply regresses predicted depth increments onto the linear slope between two known endpoint planes. More importantly, because it requires two measured endpoints along depth, it cannot be computed in the single-section or continuous holotomography settings where only one plane is measured, contradicting the claim that the same objective is used unchanged across all settings.
  2. In the continuous 3D holotomography setting, supervision is applied only after attention pooling collapses the predicted 3D slices into a single 2D plane. Because a weighted sum over depth is underdetermined by a single 2D target, high spatial autocorrelation or boundary enrichment only shows that predictions follow 3D image contrast, not that the unmeasured depth profiles are molecularly accurate.
  3. The 2D control simultaneously replaces the 3D volume with a maximum-intensity projection and removes the 3D convolution and depth-attention layers, confounding information loss from projection with architectural changes.
Questions for authors

Since the bridge objective requires two measured planes to define the target slope, how is it computed in the single-section protocol and in the continuous holotomography setting where only one 2D plane is measured? Also, given that it only regresses increments onto a deterministic linear slope, why frame it as a Schrodinger bridge?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.