acceptodds
Under review as a conference paper at ICLR 2027

Per-spot semantic conditioning for H&E to spatial-transcriptomics generation

17 views

Abstract

Predicting spatial transcriptomics (ST) from hematoxylin and eosin (H&E) images is typically performed using image encoders coupled with regression models trained under mean-squared error (MSE). Under MSE, the optimal predictor is the conditional mean, whose variance is necessarily lower than that of the true conditional distribution. We show that this structurally limits the dynamic range and spatial structure that regression can recover, not because of insufficient model capacity but because of the learning objective itself. Across HEST-bench tasks, regression compresses the predicted dynamic range to 0.74× that of the ground truth on average, with the strongest compression on the hardest tasks, while Pearson correlation largely hides this failure by remaining insensitive to spatial over-smoothing. This limitation persists even in high-capacity models and can be characterized through an aleatoric uncertainty decomposition, suggesting that H&E-to-ST models should be evaluated on dynamic range and spatial structure alongside correlation. Motivated by this limitation, we reformulate H&E-to-ST prediction as conditional generation over the Visium spot grid. Our framework introduces user-specified per-spot semantic conditioning, enabling expression to be steered along biological axes that are not recoverable from morphology alone. On a morphology-invisible interferon-response axis (image predictability r=0.06), we achieve accurate directional control while evaluating only on genes disjoint from those defining the conditioning signal. A continuous per-spot scalar consistently outperforms a tertile one-hot representation. When the control axis is instead predicted from H&E, its correlation with the target remains low (r=0.13) and controllability collapses, demonstrating that the conditioning signal must be explicitly specified. Finally, we couple conditional generation with a regression anchor that preserves positional fidelity while allowing generation to restore biologically plausible spatial variation suppressed by regression. On held-out tissues, the proposed model matches regression-level localization while improving spatial-structure fidelity beyond what Pearson correlation captures.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Per-spot semantic conditioning for H&E to spatial-transcriptomics generation

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_3xAWOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies dynamic-range compression in H&E-to-spatial-transcriptomics regression and proposes a controllable, regression-anchored flow generator. A spot-grid Transformer uses image features and optional per-spot semantic scalars through spatially adaptive normalization. An MSE-plus-MMD regression anchor provides positional information, while a residual flow is intended to recover amplitude and spatial structure. Experiments use HEST-bench and 12 liver slides. Reported results suggest improved GSC/SSC from residual generation and directional control of expression axes using externally specified labels. LABEL and EVAL genes are disjoint, and predicted labels are shown to fail on morphology-invisible axes. However, the submission lacks completed structural comparisons, consistent task aggregation, and sufficient implementation detail.

Critical strengths
  1. A useful evaluation concern. Separating localization, amplitude, and spatial structure is valuable. Pearson correlation's invariance to positive global rescaling makes the dynamic-range diagnosis relevant, and the reported mean standard-deviation ratio of 0.74 motivates additional evaluation axes.
  2. Semantic-control safeguards. The LABEL/EVAL gene separation, explicit reporting of a leaky control, and failure analysis of image-predicted labels are useful. The continuous LABEL scalar gives EVAL-gene PCC 0.118 and direction 0.97, versus the excluded leaky PCC of 0.308.
  3. An interpretable division of work. The regression anchor and residual generator have distinct intended roles. The paper candidly reports that flow does not improve held-out positional accuracy and acknowledges that morphology-invisible axes require external semantic inputs.
Critical weaknesses
  1. The submission is incomplete and difficult to reproduce. Table 1 explicitly states that STFlow/Stem structure metrics are still being computed (p. 6), while unresolved references include SEPAL and ControlNet (pp. 2–3). The ten-page PDF has no appendix with full architecture, optimizer, gene-list, metric, or preprocessing specifications. These are substantive omissions for assessing the claimed structural advance.
  2. The principal comparison does not establish superiority. Different methods omit different tasks because of memory limits, so their macro PCC values are not directly comparable. Table 1 uses one seed and lacks uncertainty and per-task results. Its missing generative-baseline GSC/SSC values are especially consequential because structure, rather than PCC, is the claimed generative benefit. Multi-encoder/context contributions are also not separated from the MMD objective.
  3. Added-input utility and theoretical claims are insufficiently established. The architecture adapts established spatial conditioning and residual-generation ideas, while external labels supply information unavailable to image-only baselines. No simple label-aware regression/template baseline establishes why flow is needed. Low residual correlation does not by itself prove an irreducible aleatoric ceiling. The label and anchor are evaluated separately, with their joint optimization left to future work (Section 5, p. 7).
Questions for authors
  1. Can all methods be evaluated on the exact common task intersection, with completed GSC/SSC results for STFlow and Stem, per-task values, and multiple training seeds?
  2. What are the exact LABEL/EVAL gene sets, axis construction, normalization, sample counts, directional-accuracy definition, MMD coefficient, model dimensions, optimizer, schedule, and model-selection protocol?
  3. How does the method compare with a label-aware regressor or a calibrated gene-program template added to the anchor, using the same image features, labels, and spatial context?
  4. What evidence distinguishes irreducible uncertainty from model or data limitations in the proposed aleatoric decomposition? Can the authors calibrate uncertainty and test the claimed ceiling on independent data?
  5. What deployment source supplies per-spot labels for morphology-invisible axes, and does a jointly anchored, label-conditioned model retain accuracy and control under noisy, missing, or spatially perturbed labels?
Reviewer_BXvmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents that models predicting spatial transcriptomics from H&E images are usually trained with an MSE loss. That loss makes them predict an average expression, which flattens high and low values and smooths out spatial patterns. The authors also argue that Pearson correlation, the standard metric, hides this problem.

The authors proposed a flow-matching generator over the spot grid that refines the output of a frozen regression model. The regression model is trained with an extra distribution-matching term (MMD) so that its predictions are less flattened. The main contribution is a per-spot conditioning signal that a user specifies, which lets the model steer expression along biological axes that are hard to see in the tissue morphology. The authors evaluate the method on HEST-bench and test controllability on liver zonation from the full HEST-1K.

Critical strengths
  • The paper discusses the over-smoothing problem of the MSE objective and the limitation of Pearson correlation in capturing amplitude collapse, and recommends reporting dynamic range and spatial-structure metrics alongside correlation. This is a useful point for the community.
  • The per-spot semantic conditioning is a reasonable direction: a spatially varying control channel targets expression axes with low image predictability, which image-only generators cannot steer.
Critical weaknesses

Limited methodological novelty: Rectified flow, SPADE-style per-spot normalization, residual anchoring, and an MSE+MMD loss all exist.

Unclear method description:

  • Several notations does not match between Figure 1 and the text, and several notations in the text are not defined.
  • LABEL and EVAL genes are not listed in the text.
  • The main contribution of the paper, computing per-spot semantic conditioning from LABEL genes is not described.

Limited evaluation: The overall benchmark gains and controllability claim are not substantiated by the current evaluation.

  • One benchmark dataset (HEST-bench hvg50) and few baselines. STPath appears only in the text, ST-Net, HisToGene, and Hist2ST are not compared, and baseline implementation details are missing.
  • Not head-to-head: PCC is reported over each method’s runnable tasks. This makes the comparison unclear in Table 1.
  • No confidence intervals or significance tests in Table 1 and 2.
  • Controllability claim is not substantial from analyzing only 4 slides for testing, out of 12 liver slides.

I appreciate the honesty in reporting the limitations, but the claims of the paper cannot be substantiated without addressing them.

Questions for authors
  • What is the LABEL and EVAL genes for each axis (liver zonation, IFN, and Wnt)?
  • How is the per-spot label computed from the LABEL genes, and how is controllability evaluated on the EVAL genes?
  • How do you rule out co-regulation between LABEL and EVAL genes? Disjoint gene sets do not by themselves establish non-circularity if the genes are co-expressed along the same axis.
  • Can you report the metrics in Table 1 for all methods on all tasks, for a head-to-head comparison?
  • How are the “direction” metric, GSC/SSC, and the Moran ratio defined, and how many axes and genes enter the direction score?
  • Please align the Figure 1 notation with the text, and define all notation in the text.
  • Please report a statistical comparison (multiple seeds, confidence intervals, or significance tests) between the proposed method and the baselines in Tables 1 and 2.
  • Can you add further baselines, such as ST-Net, HisToGene, Hist2ST, STPath, and FLAG?
  • How was τ =0.5 selected in Table 1?
  • What does footnote c in Table 1 refer to?
Reviewer_BaoZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper predicts spatial transcriptomics from H&E images and makes three claims. First, it argues that MSE regression compresses the dynamic range of predicted expression (mean std-ratio of 0.74 across HEST-bench) and that Pearson correlation cannot detect this. It supports the argument with an aleatoric decomposition of the regression residual. Second, it proposes a rectified-flow generator over the Visium spot grid with per-spot adaptive layer-norm conditioning. A user-specified per-spot scalar label, computed from one gene set and evaluated on a disjoint set, steers expression along liver zonation, interferon-response, and Wnt axes. Third, it anchors the generator to a frozen regression prediction trained with MSE plus a per-gene MMD term. The flow generates the residual, and the authors report PCC 0.446 for the anchor and improved GSC/SSC when the residual is added.

Critical strengths

The paper points out that Pearson correlation is insensitive to the scale of predictions, which is a fair concern about how H&E-to-ST methods are usually evaluated. Conditioning on a per-spot label to reach expression axes with little morphological signal is an interesting direction. The use of disjoint LABEL and EVAL gene sets, and the negative result when the label is predicted from H&E, are sound experimental choices. The authors also verify patient-level separation of the HEST-bench splits and are candid that the anchor does not exceed a regression ceiling on PCC.

Critical weaknesses

The central diagnosis treats expected behavior as a failure. For a calibrated conditional-mean predictor, the std-ratio equals its Pearson correlation with the target, so a std-ratio well below 1 is what a good point estimator should produce at PCC around 0.4, and inflating amplitude must increase expected squared error. The aleatoric decomposition in Figure 3 does not support the claimed ceiling either, because r_sto ≈ 0 holds by construction for any generator whose sampling noise is independent of the realized test noise. The baselines stop at STFlow and Stem, while more recent methods (STPath, HistoPrism from ICLR 2026, MCToGene from CVPR 2026) are not compared, and numbers from HistoPrism and STPath appear in the paper without a citation or a row in Table 1. The evidence for the anchor is confounded and incomplete. The anchor’s PCC gain over the single-encoder Ridge probes conflates the MMD objective with a concatenated multi-encoder feature stack and neighbor pooling, and no λ=0\lambda = 0 ablation or multi-encoder Ridge baseline separates the two. All results are single-seed, the GSC gain over UNI+Ridge is about 0.01 (0.543 vs. 0.534), and structure metrics for the generative baselines are reported as still being computed. The method is internally inconsistent and cannot be reproduced as written. Section 3.4 describes the anchor as the conditional mean from a single forward pass at t = 0, then trains it with an MMD term so that it is not the conditional mean. The residual flow appears to be trained on in-sample anchor residuals, and the readout y^=r+τ∗vres\hat{y} = r + \tau*v_res uses a velocity symbol for a sample, omits de-standardization, and fixes τ=0.5\tau = 0.5 without stating how it was chosen. Several components appear only in figures or not at all, including the BCE hurdle in Figure 5, the STVT encoder, the value of λ\lambda, and the label formula, and no code is provided.

Questions for authors

In addition to the major issues, could you clarify the following:

The manuscript contradicts itself in several places. Section 3.2 describes kNN-masked local attention while Table 1 attributes an OOM to dense attention, and the aleatoric analysis is described as covering three HEST-bench tasks but is plotted on liver zonation. Section 4.4 reports expanding training from 8 to 11 slides even though 4 of the 12 liver slides are held out for testing. Missing references. HistoPrism, STPath, FLAG (the source of the GSC and SSC metrics), and the STVT encoder are used or discussed without citation. SEPAL and ControlNet appear as unresolved “(?)” citations, and one reference lists “Spatia authors” in place of author names. No complexity analysis. The paper reports out-of-memory exclusions for its own model and for STFlow but gives no parameter counts, training or inference time, or memory usage. Scalability is a primary design goal of STFlow, so this comparison is necessary to evaluate the method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.