acceptodds
Withdrawn from ICLR 2027

Is H&E Image-to-Spatial Transcriptomics Simpler Than It Looks?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting spatial gene expression from routine H&E histology offers a scalable route toward spatial molecular profiling. Recent work has pursued increasingly sophisticated architectures to capture spatial context and richer expression structure. At the same time, simple estimators have shown strong performance in several studies, but what they already solve and where additional complexity is needed remain unclear. We study this behavior through the structure of prediction error under the mean-squared error (MSE) objective. Differences in average expression across genes can account for a substantial part of aggregate prediction performance, while a key unresolved error lies in recovering variation within each slide. Decomposing MSE into slide-level and within-slide components, we find that the within-slide component has lower residual-normalized parameter sensitivity in controlled neural experiments. This motivates Component-Guided Loss (CGL), which increases supervision of the within-slide component. CGL-Linear is a closed-form affine instantiation that achieves overall state-of-the-art performance across HEST-1k cohorts and gene-panel sizes. The same within-slide supervision improves existing neural models. These results suggest that substantial gains can come from aligning the training objective with prediction-error structure rather than increasing model complexity.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Spatial transcriptomics, Computational pathology, Gene expression prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3xySOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper separates prediction error into slide-level and within-slide components and proposes Component-Guided Loss (CGL) to emphasize within-slide variation. CGL-Linear is evaluated on ten HEST-1K cohorts, and CGL is also added to several neural predictors.

Critical strengths
  1. The paper shows that aggregate correlation may conceal poor within-slide spatial recovery.
  2. The benchmark covers ten cohorts, three encoders, four gene panels, and several recent methods.
  3. CGL is inexpensive and architecture-independent.
Critical weaknesses
  1. The main claims are overstated. CGL mainly adds weight to slide-centered residual error, making it close to target centering or component-wise loss reweighting. The experiments show improved within-slide reconstruction, but they do not establish a general optimization principle, a fundamentally new modeling approach, or broad conclusions about model complexity. The state-of-the-art claim should therefore be stated more cautiously.

  2. Even the ordinary Linear baseline without CGL outperforms HEXST, STFlow, and BLEEP on several reported metrics. This raises concern that the frozen image representation or benchmark protocol contributes more than the proposed loss, or that some baselines are insufficiently reproduced or tuned. Reproduction details, convergence results, method-specific hyperparameter selection, and matched tuning budgets are needed before attributing the ranking to CGL.

  3. The evaluation uses one fixed train, validation, and test split, with only two or three validation or test slides in several cohorts. Repeating neural training with different seeds measures optimization variability, not sensitivity to patient or slide selection. Patient-level K-fold cross-validation should be reported with fold-wise means and standard deviations to support the stability of the results.

  4. The effect of the component weight is not sufficiently analyzed. Results across a wider range of weights should report both slide-level and within-slide losses, together with aggregate and local metrics. This would show whether better within-slide reconstruction sacrifices slide-level prediction and whether the selected value is stable across cohorts and gene-panel sizes.

  5. The motivation for CGL needs clearer biological justification. ST data are inherently heterogeneous and highly variable. The sensitivity analysis in Eq. (3) motivates decomposing MSE and increasing the weight of within-slide expression errors by a factor of 1 + λ, but it does not establish why this weighting better captures biologically meaningful variation. Spatial variation within each gene is important, yet relationships among genes at the same location also matter. The proposed decomposition does not explicitly account for these dependencies. The paper should explain why this statistical reweighting is appropriate for ST prediction and provide evidence that it preserves meaningful gene relationships, beyond improving aggregate reconstruction metrics.

  6. Equation numbering and formatting need to be standardized. The displayed equations in Section 4 lack numbers, and the main text and appendix do not follow a clear, consistent convention for numbering and presenting formulas. This makes the mathematical presentation difficult to follow and reference. A consistent numbering policy and uniform equation formatting would improve readability.

Questions for authors

No additional questions.

Reviewer_sogAOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper decomposes H&E-to-spatial-transcriptomics prediction error into slide-level and within-slide components and proposes Component-Guided Loss (CGL) to emphasize within-slide variation. Experiments with a closed-form linear predictor and existing neural models demonstrate improved spatial expression reconstruction without increasing model complexity.

Critical strengths
  1. The slide-wise error decomposition provides a clear explanation of why strong aggregate scores can coexist with poor recovery of within-slide expression variation.
  2. Experiments span ten organ cohorts, three image encoders, and four gene-panel sizes. Improvements in existing neural models further support the usefulness of the proposed supervision.
Critical weaknesses
  1. Since SEPAL and HEXST already supervise expression deviations, CGL’s contribution appears mainly incremental, introducing direct slide-wise reweighting based on a standard error decomposition and weighted least-squares formulation.
  2. The sensitivity imbalance is examined only in a small MLP with two encoders, and a capacity-matched baseline that separately predicts slide means and within-slide residuals using standard MSE would help clarify the specific benefit of reweighting.
  3. Although held-out performance improves, within-slide deviations may contain technical noise as well as biological variation, so noise-perturbation or count-downsampling experiments would help establish robustness to measurement quality.
  4. Since Local R² can improve through changes in prediction amplitude, a validation-fitted rescaling baseline and within-slide correlations would help clarify whether CGL also predicts spatial patterns more accurately.
Questions for authors
  1. Could you clarify CGL’s main methodological advance over the deviation supervision already used in SEPAL and HEXST?
  2. How does CGL compare with a capacity-matched baseline that separately predicts slide means and within-slide residuals using standard MSE?
  3. Does CGL retain its advantage over standard MSE under controlled noise perturbation or count downsampling?
  4. Do the spatial gains remain after calibrating baseline prediction amplitudes on validation data, and are they also reflected in within-slide correlations?
Reviewer_hqPuOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies spatial gene expression prediction from H&E histology images. The authors decompose mean squared error exactly into slide-mean error and within-slide variation error, showing that strong aggregate prediction scores can coexist with poor recovery of within-slide expression variation. They also analyze the parameter sensitivities of these two components in controlled neural network experiments. Motivated by this analysis, they propose Component-Guided Loss (CGL), which increases the weight on within-slide error, and derive CGL-Linear, a closed-form affine predictor on frozen image features. The paper evaluates the approach across ten organ-level cohorts from HEST-1k, three image encoders, and four gene panels, and adds the supervision term to five existing model families to assess its applicability.

Critical strengths

1. The paper identifies an important gap between aggregate prediction scores and within-slide expression recovery, supported by direct evidence.

In Table 1, MeanOnly achieves a Flat PCC of 0.6921 despite using no image information and producing no within-slide variation. The ordinary linear model further improves aggregate correlation, while its Local R2R^2 remains negative. This comparison clearly demonstrates that aggregate scores can conceal failures in spatial expression prediction. The error decomposition and evaluation perspective help reassess what models actually learn. Their value therefore extends beyond introducing another predictor.

2. The paper demonstrates substantial benefits from changing the fitting objective while keeping predictor capacity fixed.

Linear and CGL-Linear use the same affine predictor class and are both fitted by a direct solve. With UNI and 500 genes, Local R2R^2 improves from −0.0423 to 0.1137, alongside improvements in other metrics. Figure 4 further demonstrates gains across encoders and gene panels. This comparison reduces confounding from differences in model capacity and iterative convergence, providing relatively direct support for the paper’s central practical claim: existing representations contain predictive information that can be better exploited by adjusting the objective. This result makes CGL-Linear a valuable reference baseline.

Critical weaknesses

1. The source of CGL’s advantage over existing deviation supervision remains insufficiently clarified.

The paper provides reasonably clear evidence that CGL improves over ordinary MSE, but the source of its advantage relative to existing deviation supervision requires further disentangling. Specifically, Figure 6 shows that adding CGL to HEXST, which already includes deviation supervision, further improves within-slide reconstruction. This supports the complementary value of CGL, but does not establish whether the gains primarily arise from directly constraining the final predictions, preserving deviation magnitudes, or increasing the strength of supervision on expression variation.

I suggest adding two controls using the same model and training budget: first, adequately tuning the weight of HEXST’s original deviation loss; second, applying a standardized deviation loss directly to the final predictions under the same slide-wise centering setup as CGL. The first tests whether the gains mainly reflect stronger supervision, while the second helps determine whether preserving deviation magnitudes provides additional value. These comparisons would better establish the contribution of the specific design. The current experiments support its effectiveness, but do not sufficiently explain the source of its advantage.

2. There is insufficient evidence that the performance advantage generalizes consistently to other patients.

The paper achieves strong average results on a fixed patient-disjoint split, providing evidence of predictive ability on unseen patients. However, a single split is insufficient to determine whether CGL-Linear’s advantage over other methods remains stable for a different set of patients. Some organs have only two test slides, so their results may be sensitive to individual patients or slides. A large number of measurement locations within a slide cannot substitute for more independent patients. I suggest repeated patient-level splits to test whether the advantage persists, together with paired analyses of performance differences and confidence intervals using patients as the unit of analysis. For cohorts with very few test patients, additional validation on independent patients is also needed to assess the scope of the advantage more reliably.

Questions for authors

1. What is the relationship between the sensitivity analysis and the gains of the closed-form linear model?

Figures 2–3 support the existence of sensitivity differences in the controlled MLP and show that increasing the within-slide weight can improve prediction. However, CGL-Linear and ordinary ridge regression are both fitted by a closed-form solve, so insufficient iterative optimization is not an issue. Is the sensitivity analysis intended solely as motivation for the design, or is it also intended to explain the generalization gains of the closed-form estimator? If the latter, how does sensitivity imbalance in neural networks connect to the fitting trade-offs and test performance of closed-form linear estimation?

2. Could the model selection and data preprocessing details be completed?

Please provide the expression normalization, transformation, and processing procedures across data sources; the regularization search ranges and selected values for Linear and CGL-Linear; whether parameters are shared across cohorts or gene panels; and the tuning ranges and budgets for each baseline. These details are important for assessing the fairness of the comparisons and reproducing the experiments.

3. How exactly is the within-slide loss computed in the neural augmentation experiments?

Is the term computed over complete slides or estimated using mini-batches? If mini-batches are used, please specify the sampling, slide grouping, and centering procedures, and explain their relationship to Equation (5). For multi-stage models such as SEPAL, at which stages is the term added, and which parameters are updated?

4. Could the scope of the efficiency results and spatial visualizations be clarified?

Does the timing in Section 4.5 include feature extraction, data transfer, and hyperparameter selection, and do the parameter counts refer only to the downstream predictors? Does Figure 7 use consistent color normalization across methods, and could ordinary Linear and SEPAL be included? In addition, Local R2R^2 measures centered reconstruction error and does not directly assess spatial boundaries or multiscale structures.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.