acceptodds
Withdrawn from ICLR 2027

Large Language Models as Empirical Bayes Priors for Spatial Gene Expression Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting spatial gene expression from histology images typically relies on supervised models trained on paired image-transcriptomics data, which limits accuracy when training samples are scarce. We propose BasilSup (empirical BAyes Spatial Integration of Llm prior with SUPervision), an empirical Bayes framework that uses large language models as priors for spatial gene expression prediction. A multimodal LLM reasons about associations between tissue morphology and gene expression to produce zero-shot predictions that are treated as prior distributions, which are then integrated with a supervised predictor built on pathology foundation model embeddings. The framework automatically balances the two sources of information, relying more on the LLM prior when supervised signal is weak. Across 28 spatial transcriptomics benchmark tasks, BasilSup achieves the highest performance overall, outperforming up to 6 existing state-of-the-art baselines. These results demonstrate that biological knowledge extracted from LLMs provides complementary signal to data-driven histology models, offering a principled way to incorporate domain knowledge into spatial gene expression prediction. Code will be released upon acceptance.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
large language models, vision language models, spatial transcriptomics, histology

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_E5cKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents BasilSup, which combines zero-shot multimodal-LLM predictions with supervised histology-based spatial gene-expression models. Morphological clusters are ranked by the LLM to form a spatial prior, which is fused with a UNI2-h Ridge-MLP ensemble through gene-specific weights. Experiments use HEST-Bench, STimage-1K4M, and KPMP.

Critical strengths
  1. External biological knowledge may help spatial prediction when paired data are limited.
  2. The evaluation spans several tissues, species, technologies, and a large gene panel.
  3. Multiple LLM backbones and gene-level analyses provide useful initial evidence.
Critical weaknesses
  1. Equation (3) is presented as empirical Bayes, but the implementation is a weighted average of two predictions derived from the same H&E images. The LLM scores are standardized within each slide, whereas the supervised branch predicts log-transformed expression, and no calibration between these scales is described. Squared training-set PCC is also treated as precision without showing that it estimates inverse variance. These gaps weaken the Bayesian interpretation.
  2. It is unclear whether the LLM ranking is driven by the images. The prompt already contains generated descriptions of each gene, allowing the model to rely on known marker-cell associations and match them to broad tissue patterns. Image-shuffling, blank-image, text-only, and mismatched-description controls are needed to establish that the spatial prior contains information from histology rather than the text prompt alone.
  3. The supervised branch includes two strong improvements that are separate from the main idea: UNI2-h features and an average of Ridge and MLP regressors. Most baselines do not receive the same encoder or ensemble treatment, while the supervised-only model already explains much of the final performance. Ridge, MLP, and their ensemble should be reported separately, both with and without the LLM prior.
  4. The cross-validation protocol for the zero-shot branch is underspecified. Section 3.2 samples within-slide and cross-slide pairs but does not state whether clustering and LLM comparisons are rebuilt within every fold. The paper should clarify whether a test slide can be compared with training slides or other test slides. If test images jointly construct the prior, the setting is transductive and not directly comparable with inductive baselines.
  5. The interpretability analysis is partly circular. A gene receives a high weight because its zero-shot prediction correlates with training labels, and the stronger accuracy of those same high-weight genes is then used as evidence of biological meaning. Much of this pattern follows from the weight definition. Validation on held-out genes, external data, or blinded expert assessment would provide independent evidence.
  6. Figure 1 is difficult to inspect. The text is small and rasterized, graphical elements are inconsistent, and the training and inference paths are not clearly separated. The figure should be redrawn as a clean vector diagram, and the AI-use statement should clarify whether generative tools were used to create it.
Questions for authors
  1. How are the two prediction branches calibrated, and why is squared PCC a valid precision estimate?
  2. Does the LLM remain useful when images or gene descriptions are shuffled, removed, or mismatched?
  3. Are all zero-shot operations rebuilt within each fold, and what remains when using one supervised regressor?
Reviewer_VtgUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BasilSup combines a supervised Ridge/MLP ensemble on pathology foundation-model features with a zero-shot multimodal-LLM predictor. The latter compares representative image clusters for each gene, aggregates ordinal scores, and interpolates them to patches. Per-gene weights are obtained from squared positive training correlations, with an adjustment to the supervised correlation. Experiments cover seven HEST tasks, eight human and ten mouse STimage tasks, and a larger KPMP gene panel.

Critical strengths
  1. Using morphology-aware biological knowledge to complement supervised prediction is well motivated, particularly when paired samples are scarce. Cluster-level comparisons make the idea more tractable than independent LLM calls for every patch.
  2. The benchmark breadth is useful. Table 1 reports mean PCC 0.363 versus 0.332 for the supervised-only variant and 0.313 for STFlow. The paper also examines per-gene contributions and three LLM backbones, which helps distinguish the integration from one model-specific result.
  3. The manuscript includes a supervised-only ablation on HEST, additional significance analyses, an AI-use statement, and an explicit research-use limitation for clinical application.
Critical weaknesses
  1. Core gains are limited, and the LLM prior’s contribution is not isolated across all benchmarks. On HEST-Bench, the average PCC rises only from 0.332 for Supervised only to 0.363 for BasilSup, a gain of about 0.031. On PRAD, BasilSup is actually lower than Supervised only; on LIDC and CCRCC the gains are minimal. For STimage-1K4M and KPMP, no Supervised only ablation is reported, and the KPMF analysis also lacks a no-LLM-prior control. It is therefore unclear whether the improvement comes from the LLM’s biological knowledge, the stronger UNI2-h features, the Ridge plus MLP ensemble, or regularization. BasilSup leads on only 5 of 10 mouse tasks, and the different LLM backbones vary narrowly, without key controls such as random LLM scores or shuffled gene descriptions.

  2. The “zero-shot” framing is inconsistent with the overall framework, and the conclusions are overstated. The zero-shot predictor itself needs no paired labels, but the final gene-specific weights require ground-truth expression from the training set, and the supervised predictor also requires paired training data. The full BasilSup framework therefore cannot be applied directly when no paired training data are available. Claims that the method reduces dependence on costly spatial transcriptomics assays are overstated, and no systematic few-shot experiment shows that the weighting remains reliable with fewer training slides. Conflating a zero-shot predictor with a zero-shot framework is a conceptual inconsistency.

Questions for authors
  1. Supervised-only control across all benchmarks. The paper reports a Supervised-only ablation on HEST-Bench, but not on STimage-1K4M, KPMP, or the KPMF analysis. Could you provide Supervised-only results on all benchmarks so that the contribution of the LLM prior can be separated from the effects of the stronger pathology foundation model features, the Ridge plus MLP ensemble, and regularization?

  2. Controls for the LLM prior. To demonstrate that the gains reflect the LLM’s biological knowledge rather than smoothing or regularization, could you include controls such as random LLM scores, shuffled gene names or functional descriptions, image-only prompts, and text-only prompts? These controls are currently missing and are essential to the central claim.

Reviewer_bNc4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Motivation: Predicting spatial gene expression from histology is difficult when paired image–expression training data are scarce. So the proposed BasilSup uses a multimodal LLM to compare morphological clusters and generate relative expression scores, which are converted into patch-level predictions. It combines these predictions with a supervised UNI2-h Ridge–MLP ensemble using gene specific weights derived from the training correlations, which is motivated by empirical Bayes formulation. The paper reports competitive results across 25 benchmark tasks and an additional 1,000 gene KPMP evaluation.

My overall impression is that the application is well motivated and the approach could be useful, but the LLM ablation is inconsistent, the interpretation of the weighting rule is not sufficiently justified, and the evaluation does not adequately establish the claimed advantage.

Critical strengths

1. The problem and proposed source of additional information are well motivated

The idea of using biological knowledge to support histology expression prediction is plausible. Morphology can reveal tissue compartments and cell populations associated with particular expression patterns, while paired spatial transcriptomics data may be limited.

Asking an LLM to make relative comparisons between tissue regions is also a reasonable design choice, as it avoids requiring the model to produce calibrated expression values directly.

2. The method is simple and modular

I appreciate that the pipeline combines existing components without requiring a complicated new supervised architecture. Clustering patches and selecting representative tiles reduces the number of LLM comparisons, and gene-specific weighting provides a practical way to limit the influence of an unhelpful LLM prediction.

3. Broad evaluation

The experiments cover human and mouse tissues, multiple types of tissue, different sequencing platforms, and a larger gene panel. The main descriptive results are competitive.

Critical weaknesses

1. (Major) significant numerical inconsistency

I checked the improvement and Table 1 reports average PCCs of 0.363 for BasilSup and 0.332 for the supervised-only predictor, saying an improvement of approximately 0.031. The mean from the seven printed task differences gives 0.0307. However, Appendix C instead reports a mean improvement of 0.014 over the same 350 HEST gene–task pairs.

Meanwhile, the PAAD improvements in Figure 3b are concentrated near zero, whereas Tab. 1 gives a PAAD improvement of 0.111, from 0.384 to 0.495.

2. The implemented precision” estimates are insufficiently justified

The Normal–Normal posterior mean in Eq.3 is correct under the assumptions in Eq 1 and 2. The concern is the clipped, squared training PCC for Gaussian precision.

To my knowledge, in the stated Gaussian model, precision represents inverse prediction-error variance. Correlation does not determine this quantity, a prediction can have high correlation with the target while having substantial bias or an incorrect scale. Multiplying a prediction by a positive constant makes PCC unchanged but can greatly change its prediction error. So Eq.4 is more like a heuristic reliability score to me.

3. The correction for supervised training optimism does not match the model

The paper points out that evaluating the supervised predictor on its training data produces optimistic correlations. However, its correction multiplies training PCC by a factor based on 256 principal components divided by the number of training patches (Eq.6 narratives).

But the supervised predictor is an ensemble of Ridge regression and an MLP. The number of Ridge input components is not the effective complexity of this ensemble. Even for Ridge alone, effective model complexity depends on the regularization strength and feature covariance.

Second, the number of patches does not adequately represent the amount of independent evidence for generalization to new slides or patients.

So Eq. 6 does not establish that the resulting weights reflect out-of-sample predictor quality. A model can retain a high corrected training correlation but still weak generalization.

Questions for authors
  1. Why were SKCM, HCC and LUNG excluded from HEST-Bench, and why is STFlow absent from Table 6?
  2. Is PCC computed per test slide or pooled? Are y^zs\hat y^{zs} and y^su\hat y^{su} put on a common scale before Eq. 3?

Comments on the paper

1 comment

I hope the author can respond to our concerns before the deadline; otherwise, I will reconsider my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.