- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Summary: The authors introduce TEMPUS, a multi-model architecture for predicting tissue dynamics through predicting spatial expression from H&E histology at unseen timepoints. The authors describe the issue of Spatial Transcriptomics being too expensive to collect measurements densely across multiple timepoints and hence aim to solve the issue by mapping the H&E histology to the spatial expressions. The authors evaluate TEMPUS on two synthetic datasets they curate in order to show that the model uses cross-time information and four datasets from the Visium and Xenium platform. TEMPUS shows promising results when evaluating held-out timepoints and in-slide imputation.
Critical strengths
-
The paper is well written, easy to understand and follow with a clear schematic describing the architecture.
-
The sparse history selection idea is interesting where the model only selects the top K set which is a list of past cells the model actually used for the dynamics prediction. The use of it for interpretability and efficiency is useful.
-
Experiments across a variety of datasets show the robustness of the model, and the comparisons against the strong baselines make the results more convincing.
Critical weaknesses
-
It seems unclear whether the training and evaluation come from the same tissue sections or donors. If the splits come from the same sections then spatial autocorrelation lets neighboring spots leak between train and test.
-
Evaluation seems limited to only reconstructing held-out timepoints that lie between available ST timepoints. It would be useful to see some extrapolation results where it could show if the H&E adds information beyond the existing ST data.
-
It is not clear if the datasets from Visium and Xenium technology have non-linear complex time expressions in the data. If it does not then a simple linear interpolation could easily capture the intermediate timepoints that are not captured through experiments.
-
The metrics used are PCC, MSE, and MAE which favor over-smoothed predictions since predicting values near the mean gives lower error. It could be interesting to look at the distributional distances at held-out timepoints using Wasserstein distance or MMD. Additionally, you could also look at the ratio of predicted to true variance per gene.
-
Computational complexity of the method has not been discussed.
Questions for authors
-
Are there non-linear complex time expressions in the datasets being used in the experiments? A figure or an explanation of this would help.
-
Is the model able to train on one tissue section or donor and infer on another tissue section or donor so as to prevent data leakage?
-
Have the authors considered comparing to pathology foundation models such a CONCH [1] and UNI [2] which are pretrained on large histopathology corpora?
[1] Chen, Richard J., et al. "Towards a general-purpose foundation model for computational pathology." Nature medicine 30.3 (2024): 850-862.
[2] Lu, Ming Y., et al. "A visual-language foundation model for computational pathology." Nature medicine 30.3 (2024): 863-874.
7 comments
Response to Reviewer 2U4y (2/4)
W3 + Q1. Non-linear temporal dynamics in the data
Yes, the datasets contain strongly non-linear dynamics. We added this analysis to the paper (Appendix K, Figure 17; changes in blue, we have updated the paper please refer to it for the figures). We tested it directly on the data, without any model, on the datasets with several sections per time point, using section-to-section variation within a time point as the noise reference:
- Time-variable genes: expression differs between time points (ANOVA, FDR < 0.05).
- Non-linear: a straight line in time is rejected (lack-of-fit F-test).
- Interpolation test: each time point is held out and predicted by straight-line interpolation of its neighbours. If change were linear, only 5% of genes would miss.
| Dataset | Time-variable genes | Non-linear | Shape | Missed by linear interpolation (5% expected) |
|---|---|---|---|---|
| Human heart (Visium; 38 sections, 9 time points) | 3,546 / 5,664 | 78% | 75% not captured by a quadratic | 35% |
| Developing pancreas (Xenium; 12 sections, 6 time points) | 772 / 1,850 | 69% | 84% transient peak or dip | 47% |
Most temporal changes are non-linear and often transient. Transient genes rise to a peak (or fall to a dip) at an intermediate time point and then return, and genes not captured by a quadratic change direction more than once. Examples are NPPA, HEY2, IRX4 and TNNI3 in the heart, and SOX9, KRT19, COL1A1 and SST in the pancreas (Figure 17, right).
Consistent with this, simple temporal fits fall well short of TEMPUS on Xenium (Table 10): linear extrapolation reaches 0.376, spline extrapolation 0.337, PCA interpolation 0.382 and temporal k-NN 0.454, against 0.535 for TEMPUS. All of these baselines use the same UNI features, TERRA space and frozen decoder, so they differ from TEMPUS only in how the prediction is formed from the history. TEMPUS itself makes no parametric assumption about time: it retrieves from all past time points, conditioned on their continuous time gap to the target.
Response to Reviewer 2U4y (3/4)
W4. Distributional metrics
We agree that PCC, MSE and MAE favour smooth predictions. We now evaluate distributional distances at the held-out time points of the developing human pancreas Xenium dataset (single cell; 13, 14 and 17 pcw held out): the ratio of predicted to measured variance per gene, the per-gene Wasserstein distance (normalised by the measured sd), and the MMD between predicted and measured expression (on 30 principal components).
All methods predict expected expression, so each prediction is sampled from the count model of the shared decoder. The noise model is identical for every method, so no method is penalised for lacking technical noise.
| Method | Variance ratio (1 = ideal) | Wasserstein ↓ | MMD ↓ |
|---|---|---|---|
| TEMPUS | 0.95 | 0.095 | 0.066 |
| TEMPUS (no memory) | 0.93 | 0.101 | 0.066 |
| BLEEP | 0.85 | 0.119 | 0.061 |
| MERGE | 0.82 | 0.128 | 0.096 |
| NicheFlow | 0.88 | 0.131 | 0.274 |
| NicheFlow+X | 0.94 | 0.105 | 0.142 |
| Upper bound | 1.00 | 0.040 | 0.013 |
True expression embeddings decoded with the same decoder. Mean over 3 seeds and held-out time points.
TEMPUS is closest to the measured distribution: its per-gene spread matches the measurements best (ratio 0.95), and it has the lowest Wasserstein distance (0.095 vs. 0.105 to 0.131 for the baselines). Its MMD is on par with BLEEP (0.066 vs. 0.061) and well below MERGE and both generative NicheFlow models. Without sampling, the mean prediction of every method is smoother than the measurements, including the upper bound. A generative output head conditioned on the temporal memory is a natural extension, and we will add these metrics to the paper.
W5. Computational complexity
We thank the reviewer. The analysis is in Appendix H.8 (Table 15), and we will add a summary to the main text. Because retrieval is confined to bounded local crops, inference is linear in the number of cells and its memory is independent of slide size: O(NT(M+K)d) compute and O(N(K+T)d) memory, with N target cells, T history time points, M cells per history crop, K = 32 selected cells per time point and d = 128. Measured on one A100 GPUS:
| Slide size (cells) | Crops | Total time | Peak GPU memory |
|---|---|---|---|
| 10,000 | 10 | 57 ms | 0.12 GB |
| 50,000 | 50 | 0.28 s | 0.12 GB |
| 190,000 (whole Xenium section) | 190 | 3.6 s | 0.12 GB |
A whole 190,000-cell Xenium section completes in about 3.6 s with a flat memory footprint, at a scale where the published single-section graph baselines do not run (Table 7).
Response to Reviewer 2U4y (1/4)
We thank the reviewer for the positive assessment of the clarity of the paper, the sparse history selection and the breadth of the experiments, and for the precise questions on evaluation. We answer each point below. Unless stated otherwise, results are count-space values, mean ± sd over 3 seeds.
W1 + Q2. Train/test splits, donors and spatial autocorrelation
Cross-time completion (Tables 1, 2 and 7). We split by time point, not by spot. Every section from a test time point is held out in full, so no spot, neighbourhood or expression value of a test section is seen during training. Each donor (or animal) is profiled at a single age, so held-out time points are also held-out individuals. The model and the shared count decoder are fitted on training sections only. At test time, the memory holds training sections and, for later test weeks, the model's own earlier predictions (Appendix B), never measured test expression.
In-slide imputation (Table 3). Here a contiguous region covering 90% of the cells of the test section is held out, and the remaining 10% is given as a prompt, exactly as for PH2ST. Metrics are computed on the held-out region only. Neighbouring information from the prompt is therefore part of the task definition, and it is available equally to both methods.
Cross-platform transfer (new). We also train on one platform and test on another. TEMPUS is trained only on the Xenium fetal pancreas atlas (8–17 pcw) and tested on the Visium PCW 20 pancreas section of the paper. The two datasets differ in study, donors, technology and resolution; the UNI features, TERRA targets and count decoder are the same as in the paper.
When trained only on another platform, TEMPUS keeps 94% of its PCC, and its MAE improves.
W2. Extrapolation, and whether H&E adds information beyond ST
We thank the reviewer and clarify the protocol: all cross-time results in the paper are extrapolation. Each model is trained only on early time points and evaluated on later time points it has never seen.
Interpolation (new). We have now also evaluated interpolation on heart, hiding 1, 3 or 5 consecutive weeks. With 5 hidden weeks, only PCW 6.0, 7.0, 7.5 and 12.0 are observed. In this setting the memory contains observed sections on both sides of the target; no architectural change is needed, since time enters the model only through the continuous time gap Δτ.
HVG50: gene-wise PCC across spots, averaged over the top 50 highly variable genes.
TEMPUS improves over the no-memory model at every gap, most clearly on the gene-wise metric. The spot-wise gain is smaller than in extrapolation, as expected: with observations on both sides of the target, much of the spot-wise signal is already explained by H&E.
H&E adds information beyond the existing ST. Table 10 ablates the two inputs of TEMPUS on Xenium: removing the target H&E lowers PCC from 0.535 to 0.412, and removing past ST lowers it to 0.472. Both inputs are needed, and the target H&E carries the larger share.