acceptodds
Under review as a conference paper at ICLR 2027

TEMPUS: TEmporal Modeling Platform for Unified Spatial Omics

Abstract

Tissues evolve over time through development, homeostasis, and disease, and understanding these dynamics requires measurements that are spatially resolved and temporally informative. Spatial transcriptomic studies increasingly profile tissues at multiple time points alongside hematoxylin and eosin (H&E) histology, but the cost of the two modalities is disproportionate: H&E is acquired routinely while spatial transcriptomics remains expensive and is profiled at only a few time points. We formulate tissue dynamics prediction as a multi-modal cross-time task: predict spatial gene expression at any time point given dense H&E images across all time points and sparse spatial transcriptomics at few. We introduce TEMPUS (Temporal Modeling Platform for Unified Spatial Omics), the first model for this task. TEMPUS combines three design choices: (i) a learned per-query top- selection over a two-dimensional grid of past observations indexed by time point and tissue coordinate, compressed into one per-query, per-time-point token consumed by a cross-attention decoder; (ii) a chained cross-modal and temporal objective with cycle-consistency, where an auxiliary decoder reconstructs each pre-fusion history token from the prediction-time latent, preventing collapse onto the most recent observation; and (iii) operation in frozen foundation-model embedding space, decoded to counts only at evaluation, which keeps the model gene-panel and platform agnostic and scales to whole-tissue in Xenium resolution. On two synthetic benchmarks where the target provably cannot be predicted from any single time point, and four experimental datasets, the mean Pearson correlation improves over the strongest baselines from non-temporal histology-to-expression methods and temporal models. Ablations validate that learned top- outperforms -nearest-neighbor retrieval, and that per-query attention is interpretable against ground-truth niche identity.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Spatial transcriptomics, multimodal learning, spatiotemporal representation learning, cross-modal prediction, computational biology

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2U4yOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Summary: The authors introduce TEMPUS, a multi-model architecture for predicting tissue dynamics through predicting spatial expression from H&E histology at unseen timepoints. The authors describe the issue of Spatial Transcriptomics being too expensive to collect measurements densely across multiple timepoints and hence aim to solve the issue by mapping the H&E histology to the spatial expressions. The authors evaluate TEMPUS on two synthetic datasets they curate in order to show that the model uses cross-time information and four datasets from the Visium and Xenium platform. TEMPUS shows promising results when evaluating held-out timepoints and in-slide imputation.

Critical strengths
  • The paper is well written, easy to understand and follow with a clear schematic describing the architecture.

  • The sparse history selection idea is interesting where the model only selects the top K set which is a list of past cells the model actually used for the dynamics prediction. The use of it for interpretability and efficiency is useful.

  • Experiments across a variety of datasets show the robustness of the model, and the comparisons against the strong baselines make the results more convincing.

Critical weaknesses
  • It seems unclear whether the training and evaluation come from the same tissue sections or donors. If the splits come from the same sections then spatial autocorrelation lets neighboring spots leak between train and test.

  • Evaluation seems limited to only reconstructing held-out timepoints that lie between available ST timepoints. It would be useful to see some extrapolation results where it could show if the H&E adds information beyond the existing ST data.

  • It is not clear if the datasets from Visium and Xenium technology have non-linear complex time expressions in the data. If it does not then a simple linear interpolation could easily capture the intermediate timepoints that are not captured through experiments.

  • The metrics used are PCC, MSE, and MAE which favor over-smoothed predictions since predicting values near the mean gives lower error. It could be interesting to look at the distributional distances at held-out timepoints using Wasserstein distance or MMD. Additionally, you could also look at the ratio of predicted to true variance per gene.

  • Computational complexity of the method has not been discussed.

Questions for authors
  • Are there non-linear complex time expressions in the datasets being used in the experiments? A figure or an explanation of this would help.

  • Is the model able to train on one tissue section or donor and infer on another tissue section or donor so as to prevent data leakage?

  • Have the authors considered comparing to pathology foundation models such a CONCH [1] and UNI [2] which are pretrained on large histopathology corpora?

[1] Chen, Richard J., et al. "Towards a general-purpose foundation model for computational pathology." Nature medicine 30.3 (2024): 850-862.

[2] Lu, Ming Y., et al. "A visual-language foundation model for computational pathology." Nature medicine 30.3 (2024): 863-874.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 2U4y (1/4)

We thank the reviewer for the positive assessment of the clarity of the paper, the sparse history selection and the breadth of the experiments, and for the precise questions on evaluation. We answer each point below. Unless stated otherwise, results are count-space values, mean ± sd over 3 seeds.

W1 + Q2. Train/test splits, donors and spatial autocorrelation

Cross-time completion (Tables 1, 2 and 7). We split by time point, not by spot. Every section from a test time point is held out in full, so no spot, neighbourhood or expression value of a test section is seen during training. Each donor (or animal) is profiled at a single age, so held-out time points are also held-out individuals. The model and the shared count decoder are fitted on training sections only. At test time, the memory holds training sections and, for later test weeks, the model's own earlier predictions (Appendix B), never measured test expression.

In-slide imputation (Table 3). Here a contiguous region covering 90% of the cells of the test section is held out, and the remaining 10% is given as a prompt, exactly as for PH2ST. Metrics are computed on the held-out region only. Neighbouring information from the prompt is therefore part of the task definition, and it is available equally to both methods.

Cross-platform transfer (new). We also train on one platform and test on another. TEMPUS is trained only on the Xenium fetal pancreas atlas (8–17 pcw) and tested on the Visium PCW 20 pancreas section of the paper. The two datasets differ in study, donors, technology and resolution; the UNI features, TERRA targets and count decoder are the same as in the paper.

Training dataPCC ↑MSE ↓MAE ↓
Visium (paper setting)0.812 ± 0.00616.32 ± 0.070.477 ± 0.018
Xenium only (cross-platform)0.764 ± 0.02521.78 ± 0.540.416 ± 0.003

When trained only on another platform, TEMPUS keeps 94% of its PCC, and its MAE improves.

W2. Extrapolation, and whether H&E adds information beyond ST

We thank the reviewer and clarify the protocol: all cross-time results in the paper are extrapolation. Each model is trained only on early time points and evaluated on later time points it has never seen.

DatasetTrainTest
Human heart (Visium)PCW 6.0–8.0PCW 9.0–12.0 (up to 4 weeks ahead)
Human pancreas (Visium)PCW 12–18PCW 20
Developing pancreas (Xenium)PCW 8–12PCW 13, 14, 17
Kidney IRI (Xenium)sham, 4 h12 h to 6 weeks

Interpolation (new). We have now also evaluated interpolation on heart, hiding 1, 3 or 5 consecutive weeks. With 5 hidden weeks, only PCW 6.0, 7.0, 7.5 and 12.0 are observed. In this setting the memory contains observed sections on both sides of the target; no architectural change is needed, since time enters the model only through the continuous time gap Δτ.

Hidden weeksTEMPUS PCCTEMPUS HVG50No memory PCCNo memory HVG50
10.848 ± 0.0020.3160.838 ± 0.0030.275
30.858 ± 0.0060.3250.845 ± 0.0030.311
50.802 ± 0.0220.3480.792 ± 0.0060.309

HVG50: gene-wise PCC across spots, averaged over the top 50 highly variable genes.

TEMPUS improves over the no-memory model at every gap, most clearly on the gene-wise metric. The spot-wise gain is smaller than in extrapolation, as expected: with observations on both sides of the target, much of the spot-wise signal is already explained by H&E.

H&E adds information beyond the existing ST. Table 10 ablates the two inputs of TEMPUS on Xenium: removing the target H&E lowers PCC from 0.535 to 0.412, and removing past ST lowers it to 0.472. Both inputs are needed, and the target H&E carries the larger share.

AuthorsOpenReview ↗

Response to Reviewer 2U4y (2/4)

W3 + Q1. Non-linear temporal dynamics in the data

Yes, the datasets contain strongly non-linear dynamics. We added this analysis to the paper (Appendix K, Figure 17; changes in blue, we have updated the paper please refer to it for the figures). We tested it directly on the data, without any model, on the datasets with several sections per time point, using section-to-section variation within a time point as the noise reference:

  • Time-variable genes: expression differs between time points (ANOVA, FDR < 0.05).
  • Non-linear: a straight line in time is rejected (lack-of-fit F-test).
  • Interpolation test: each time point is held out and predicted by straight-line interpolation of its neighbours. If change were linear, only 5% of genes would miss.
DatasetTime-variable genesNon-linearShapeMissed by linear interpolation (5% expected)
Human heart (Visium; 38 sections, 9 time points)3,546 / 5,66478%75% not captured by a quadratic35%
Developing pancreas (Xenium; 12 sections, 6 time points)772 / 1,85069%84% transient peak or dip47%

Most temporal changes are non-linear and often transient. Transient genes rise to a peak (or fall to a dip) at an intermediate time point and then return, and genes not captured by a quadratic change direction more than once. Examples are NPPA, HEY2, IRX4 and TNNI3 in the heart, and SOX9, KRT19, COL1A1 and SST in the pancreas (Figure 17, right).

Consistent with this, simple temporal fits fall well short of TEMPUS on Xenium (Table 10): linear extrapolation reaches 0.376, spline extrapolation 0.337, PCA interpolation 0.382 and temporal k-NN 0.454, against 0.535 for TEMPUS. All of these baselines use the same UNI features, TERRA space and frozen decoder, so they differ from TEMPUS only in how the prediction is formed from the history. TEMPUS itself makes no parametric assumption about time: it retrieves from all past time points, conditioned on their continuous time gap to the target.

AuthorsOpenReview ↗

Response to Reviewer 2U4y (3/4)

W4. Distributional metrics

We agree that PCC, MSE and MAE favour smooth predictions. We now evaluate distributional distances at the held-out time points of the developing human pancreas Xenium dataset (single cell; 13, 14 and 17 pcw held out): the ratio of predicted to measured variance per gene, the per-gene Wasserstein distance (normalised by the measured sd), and the MMD between predicted and measured expression (on 30 principal components).

All methods predict expected expression, so each prediction is sampled from the count model of the shared decoder. The noise model is identical for every method, so no method is penalised for lacking technical noise.

MethodVariance ratio (1 = ideal)Wasserstein ↓MMD ↓
TEMPUS0.950.0950.066
TEMPUS (no memory)0.930.1010.066
BLEEP0.850.1190.061
MERGE0.820.1280.096
NicheFlow0.880.1310.274
NicheFlow+X0.940.1050.142
Upper bound1.000.0400.013

True expression embeddings decoded with the same decoder. Mean over 3 seeds and held-out time points.

TEMPUS is closest to the measured distribution: its per-gene spread matches the measurements best (ratio 0.95), and it has the lowest Wasserstein distance (0.095 vs. 0.105 to 0.131 for the baselines). Its MMD is on par with BLEEP (0.066 vs. 0.061) and well below MERGE and both generative NicheFlow models. Without sampling, the mean prediction of every method is smoother than the measurements, including the upper bound. A generative output head conditioned on the temporal memory is a natural extension, and we will add these metrics to the paper.

W5. Computational complexity

We thank the reviewer. The analysis is in Appendix H.8 (Table 15), and we will add a summary to the main text. Because retrieval is confined to bounded local crops, inference is linear in the number of cells and its memory is independent of slide size: O(NT(M+K)d) compute and O(N(K+T)d) memory, with N target cells, T history time points, M cells per history crop, K = 32 selected cells per time point and d = 128. Measured on one A100 GPUS:

Slide size (cells)CropsTotal timePeak GPU memory
10,0001057 ms0.12 GB
50,000500.28 s0.12 GB
190,000 (whole Xenium section)1903.6 s0.12 GB

A whole 190,000-cell Xenium section completes in about 3.6 s with a flat memory footprint, at a scale where the published single-section graph baselines do not run (Table 7).

Reviewer_KkeaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces TEMPUS, a model for predicting spatial gene expression at time points where only H&E histology is available, using paired histology–spatial transcriptomics observations from earlier time points. It combines frozen UNI image features and TERRA ST expression embeddings with spatial and temporal encodings. For each target cell or spot, TEMPUS retrieves the top-K most relevant cells from each historical time point, compresses them into one token per time step, and integrates these tokens through cross-attention. An auxiliary cycle-consistency loss encourages the model to retain information across the history rather than rely only on the most recent observation. The authors evaluate TEMPUS on two synthetic benchmarks that simulate microenvironmental niches and require temporal information and four real Visium and Xenium datasets covering human development and mouse kidney injury. They report higher Pearson correlations than single-section and temporal baselines, supported by ablations of history selection, cycle consistency, and history depth. However, there are major points that question the soundness, validity and impact of the work, which have motivated my initial recommendation for this work to be reject.

Critical strengths
  • Evaluation across multiple ST modalities. The authors use four temporal datasets: human heart and human fetal pancreas acquired with the Visium technology and kidney, and developing human pancreas with Xenium. However, only three of them are reported in the main text. The datasets span spot-based and single-cell evaluation which strengthens the generalizability of the framework.
Critical weaknesses

W1. TEMPUS incorporates each spot coordinate p (2D position on the section). Each past time point is cut to a disk of radius r_h around the target and is mentioned that this "bounds the cost of the subsequent top-K selection and biases context toward spatial proximity". Could the authors clarify the role of the coordinates? If the distinct images were aligned across the temporal dimension I would understand this strategy, but it is difficult for me with the current setup. I am also not sure that tissue alignment is possible since distinct slides correspond to different subjects, e.g; on fetal heart (PCW 6 vs 12) and pancreas, the tissue shape changes a lot, so a fixed radius in raw coordinates is doubtful.

W2. The baselines are not properly setup. Different models use different vision encoders which could hide the impact from the temporal component. Supplementary Table 13 includes a match-encoder ablation for the Heart dataset but does not include Nicheflow or Nicheflow-X. We also observe that some of the baselines substantially improve when the encoder is matched to UNI2 (e.g., BLEEP: 0.671 --> 0.716). Could the authors recreate Table 2 with matched encoders for all baselines and datasets?

W3. The synthetic datasets are not sufficient for testing the behaviour for the stated application: the expression target isn't count-like, and the temporal and spatial dynamics are idealized. Moreover, some of the synthetic datasets seem to be built around the TEMPUS model. Trajectory Memory needs "exactly the inductive bias top-K over the joint x + spatial scoring is designed to deliver". For example, using orthogonal niche directions make retrieval by dot-product similarity almost trivial, so winning here partly restates the design.

W4. No visualizations of the model's predictions are shown in the main manuscript and those included in the supplemental do not closely recapitulate the ground truth expression (e.g., Figure 14 or Figure 15). For example, in Figure 14 we do see agreement for TNNI3 and TMSB10 genes in PCW6.0-8.0, which belong to the training set. However, in further timepoints (timepoints in the test set), the predictions significantly differ from the ground truth. Could the authors include PCC/MSE/RMSE metrics along with the visualization for TEMPUS as well as for other baselines, and explain the disagreement.

Questions for authors

Q1. Could the authors recreate Table 2 with matched encoders for all baselines and datasets?

Q2. Explicitly clarify that the distinct time points come from different animals/individuals. This should be made clear in the introduction since the gene expression profile will not change uniquely based on the temporal component but more prominently based on the distinct subjects.

Q3. Can the authors clarify the role of the coordinates positional encoding, the "disk of history around the target", and whether serial section alignment would be required along this feature? An ablation of the model with and without positional encoding will also be helpful.

Q4. Visualizations of gene expression, as described in W4.

Q5. Improvement on the synthetic datasets. For example, using semi-synthetic data from real sections could be an option, e.g. real Xenium or Visium cells and morphology, with a known temporal program injected. This point refers to the same comment as W3.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Kkea (1/4)

We thank the reviewer for a careful and specific review. The questions were concrete and actionable, and the semi-synthetic benchmark suggested in Q5 proved especially informative, so we have adopted it. We answer each point below. Unless stated otherwise, results are count-space values, mean ± sd over 3 seeds.

W2 + Q1. Matched encoders for all baselines

We apologise for the confusion. TEMPUS, NicheFlow and NicheFlow+X were already evaluated with the same UNI (v1) features (NicheFlow uses no image), the same TERRA target space and the same frozen decoder, which is why Table 13 did not list them. The remaining differences in Table 2 concerned BLEEP and MERGE, which we have now re-run in the matched setting on all three datasets (heart as in Table 13).

MethodHeart PCC ↑MSE ↓MAE ↓Pancreas PCC ↑MSE ↓MAE ↓Kidney PCC ↑MSE ↓MAE ↓
Single section (no memory)
BLEEP0.716 ± 0.1003.9960.3650.797 ± 0.00818.290.4480.490 ± 0.0020.1330.171
MERGE0.707 ± 0.2004.5330.3880.806 ± 0.00217.470.5760.497 ± 0.0030.1330.171
TEMPUS (no memory)0.738 ± 0.0084.2180.3310.788 ± 0.00417.050.4900.480 ± 0.0030.1320.171
Temporal
NicheFlow0.613 ± 0.0375.9300.3650.760 ± 0.00827.190.6380.431 ± 0.0130.1490.193
NicheFlow+X0.705 ± 0.0034.5290.3530.781 ± 0.00322.960.5940.463 ± 0.0140.1430.185
TEMPUS0.794 ± 0.0063.6800.3180.812 ± 0.00616.320.4770.512 ± 0.0080.1290.165

All methods: same frozen UNI features, TERRA target space and frozen decoder within each dataset; held-out time points only.

With matched encoders, TEMPUS remains the best method on all three datasets and on 8 of 9 metrics. As the reviewer noted, matching helps BLEEP on heart (0.671 to 0.716), and TEMPUS still leads it by +0.078. On the small Visium pancreas dataset, TEMPUS and MERGE are within one sd on PCC (0.812 vs. 0.806) and BLEEP has the lowest MAE; the gain is largest on heart. For completeness, Hist2ST and STFlow, matched on heart in Table 13, reach 0.724 and 0.732. We will replace Table 2 with this version.

AuthorsOpenReview ↗

Response to Reviewer Kkea (2/4)

W1 + Q3. Role of the coordinates, the history crop, and alignment

We thank the reviewer for raising this. Our description in Section 2.1 was not precise enough, and the concern is legitimate.

What the coordinates do. The spatial encoding PEₛₚ is a Fourier-feature encoding of normalised slide coordinates (Appendix C). It is added to both target and history tokens, so in the submitted configuration raw coordinates can contribute to cross-section scoring. The history crop of radius rₕ is centred at the target's coordinates in each past section. Its main role is to bound compute: inside the crop, past cells are selected by learned H&E and expression relevance, and the crop is large (rₕ = 2.5 rₜ by default, Appendix D).

Is serial-section alignment required? (new ablation) No. Since different time points come from different individuals and are not registered, we tested the model without relying on coordinates across sections:

  • No PE: PEₛₚ removed.
  • Random crop: each past crop is centred on a random cell.
  • H&E-matched crop (OT): each past crop is placed by optimal transport between the H&E patch features of the two sections, with no registration and no coordinates.
Variant (Human heart)Count PCC
TEMPUS (submitted)0.794 ± 0.006
No positional encoding0.786 ± 0.012
Random crop centre0.762 ± 0.015
H&E-matched crop (OT, no coordinates)0.809 ± 0.018

Removing the positional encoding has no measurable effect (0.786 vs. 0.794). Placing the crop at random costs 0.03 PCC, so crop placement does matter: the crop is most useful when it covers a corresponding tissue region. That correspondence does not require registration, however. The H&E-matched crop, which uses no coordinates across sections, performs at least as well as raw coordinates (0.809). TEMPUS therefore does not depend on serial-section alignment. We will add this ablation, describe the H&E-matched crop as the registration-free option, and revise the sentence in Section 2.1 stating that the crop biases context toward spatial proximity.

Q2. Time points come from different individuals

We agree and will state this in the introduction: time and individual are confounded by design, as in any developmental atlas. Three checks (new appendix figure 18 we have updated the paper in openreview so please refer to it for plots) show that the changes between time points are driven mainly by time rather than by individual differences.

  1. Age is predictable for an unseen individual (heart). A ridge model trained on the other time points predicts the age of a held-out individual with Spearman ρ = 0.97 (p = 0.005), with a mean error of 0.78 weeks against 1.88 for a constant prediction. Development therefore leaves a time signature that is shared across individuals.
  2. Time differences exceed donor differences (Xenium pancreas, 12 sections, 8 donors). Expression distance (1 − r) is 0.09 between two sections of the same donor, 0.70 between donors of the same age, and 1.18 between different ages, and it grows with the age gap (Mantel r = 0.47, p = 0.011). Donor effects are real, but time effects are larger.
  3. The memory transfers across individuals. The memory of TEMPUS contains only other individuals at earlier time points, yet it improves significantly over no memory (+0.055 PCC, p < 0.001). This gain could not arise if the differences were purely individual-specific.

Donor effects exist, but a larger temporal component is shared across individuals, and this is the component TEMPUS exploits.

AuthorsOpenReview ↗

Response to Reviewer Kkea (3/4)

We agree with the reviewer. The two synthetic benchmarks were designed as controlled sanity checks with a known ground truth: they test whether a model can use memory and whether its attention can be audited, not how it performs on real tissue. In Trajectory Memory, the approximately orthogonal niche directions make same-niche retrieval easy, so a win there partly reflects the design, as the reviewer points out. We will state this explicitly.

Following the reviewer's suggestion, we built a semi-synthetic benchmark from real data. We start from real Xenium sections of the developing human pancreas (6 time points, 8–17 pcw), keeping the real cells, coordinates, H&E morphology and counts, and inject a known temporal programme:

  • Niches: k-means (k = 6) on the real neighbourhood embedding.
  • Programme: each niche gets its own module of 30 genes, whose level follows a latent trajectory over time (AR(1), ρ = 0.8). It is non-monotonic, only partly predictable from the previous time point, and invisible in H&E, since morphology is unchanged.
  • Counts: Poisson-resampled from the modified expression, so realistic count noise is kept.

Because the true programme is known at every time point, we can test whether a model recovers the temporal change itself, not only per-spot accuracy. All methods use the same H&E features, targets and frozen decoder.

SettingTEMPUSNo memoryNicheFlow+XBLEEPNicheFlow
Forecast, count PCC (13, 14, 17 pcw unseen)0.489 ± 0.0150.451 ± 0.0120.433 ± 0.0350.409 ± 0.0280.343 ± 0.054
Interpolation, count PCC (12, 14 pcw unseen)0.599 ± 0.0030.584 ± 0.0010.522 ± 0.0310.526 ± 0.0100.357 ± 0.071
Interpolation, programme recovery (r)0.9090.4280.7840.7010.898

Programme recovery: correlation between the predicted and true programme trajectory of each niche across the hidden weeks (constant prediction: 0; oracle: 0.956).

TEMPUS is the most accurate method in both settings and recovers the injected programme close to the oracle (0.909 vs. 0.956). Without memory, recovery drops to 0.428; since the programme is invisible in H&E, this gain comes from the temporal memory. NicheFlow tracks the programme at the niche level (0.898) but loses per-spot accuracy (0.357). TEMPUS is the only method that does both. We will add the benchmark and its generator code to the appendix.

Reviewer_guBUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TEMPUS, a framework for predicting and imputing spatial gene expression under sparse temporal and spatial observations. By integrating histology images (H&E) available at different time points with spatial transcriptomics measurements from selected historical stages, TEMPUS predicts gene expression at target time points or unmeasured spatial locations. The framework employs learnable historical reference retrieval, hierarchical within-time and cross-time attention, and historical representation reconstruction to integrate morphological, molecular, and spatial information. Experiments on synthetic benchmarks and real-world datasets demonstrate the potential benefits of incorporating historical information for spatial gene expression prediction.

Critical strengths
  1. The paper proposes a target-location-driven framework for integrating historical multimodal information across time. For each target location, TEMPUS retrieves relevant historical references and employs hierarchical attention mechanisms to aggregate information within and across time points. The historical representation reconstruction objective encourages the fused representations to preserve information from multiple stages, while self-attention among target locations further captures spatial contextual dependencies. In addition, sparse retrieval and historical token compression help reduce the computational cost of subsequent attention operations, improving the scalability of the framework.
Critical weaknesses

Weaknesses

  1. The ability of TEMPUS to characterize temporal dynamics remains limited. The framework primarily integrates information from discrete historical time points without explicitly modeling continuous-time state evolution. Therefore, it remains unclear whether the model can reliably perform temporal interpolation or extrapolation, or capture meaningful rates of biological change. Moreover, the model does not directly infer gene regulatory mechanisms, limiting its ability to provide mechanistic insights beyond predictive temporal associations.

  2. The consistency of cross-modal representations is not explicitly enforced. Histology images and gene expression profiles are independently encoded and integrated through downstream prediction objectives, without an explicit cross-modal alignment constraint. It remains unclear whether this indirect alignment is sufficient for accurate historical reference retrieval and multimodal fusion. Comparisons with contrastive learning or other cross-modal alignment strategies would help clarify this issue.

  3. The experimental evaluation on real-world datasets could be strengthened. Improvements on some datasets are relatively modest, and additional statistical significance tests and cross-platform or cross-individual evaluations would help establish robustness and generalizability. Furthermore, the primary use of spot-wise PCC does not adequately measure the recovery of gene-specific spatial expression patterns. Gene-wise PCC/Spearman correlations across spatial locations, together with evaluations of spatially variable genes and marker genes, would provide a more comprehensive assessment.

Questions for authors
  1. How does TEMPUS distinguish meaningful temporal dynamics from correlations among discrete historical observations? Can the authors evaluate its ability to interpolate or extrapolate to unseen time points, particularly under irregular temporal sampling, and clarify whether the framework can capture biologically meaningful temporal changes beyond predictive associations?

  2. What motivates the decision to integrate histological and transcriptomic representations without an explicit cross-modal alignment objective? Have the authors considered contrastive alignment or other cross-modal consistency constraints, and how might these affect historical reference retrieval and generalization across datasets?

  3. Given the relatively modest improvements on some real-world datasets, can the authors provide results from multiple random seeds with statistical significance tests? Additionally, how robust is TEMPUS under cross-platform, cross-batch, or cross-individual evaluation settings?

  4. Can the authors complement spot-wise PCC with gene-wise PCC or Spearman correlations computed across spatial locations? In particular, how well does TEMPUS recover the spatial distributions of marker genes and spatially variable genes, and preserve biologically meaningful spatial expression patterns?

6 comments
AuthorsOpenReview ↗

Response to Reviewer guBU (1/4)

We thank the reviewer for the detailed review and for recognising the target-driven retrieval, the history reconstruction objective and the scalability of sparse retrieval. We answer each point below. Unless stated otherwise, results are count-space values, mean ± sd over 3 seeds, decoded through the same frozen decoder for every method.

W1 + Q1. Temporal dynamics, interpolation and extrapolation

Continuous time and irregular sampling. TEMPUS does not index time points. Each past observation enters the memory with its continuous time gap to the target, Δτ = t − τ, through a sinusoidal encoding (Section 2.2), so any spacing and any query time are supported. All our datasets are irregularly sampled: heart at PCW 6.0–12.0 in steps of 0.5 to 1 week, Xenium pancreas at 8, 11, 12, 13, 14 and 17 pcw, and kidney from 4 h to 6 weeks. We agree that TEMPUS does not model a continuous latent state (e.g., a neural ODE), and we will state this in the limitations.

Extrapolation. This is the main setting of the paper: all cross-time results (Tables 1, 2 and 7) predict later time points that were never seen in training. On heart, TEMPUS reaches 0.794 ± 0.006 against 0.738 ± 0.008 for the same model without memory (p < 0.001, see Q3).

Interpolation (new). We hide 1, 3 or 5 consecutive heart weeks. With 5 hidden weeks, only PCW 6.0, 7.0, 7.5 and 12.0 are observed. In this setting the memory contains observed sections on both sides of the target. No architectural change is needed, since time enters the model only through Δτ.

Hidden weeksTEMPUS PCCTEMPUS HVG50No memory PCCNo memory HVG50
10.848 ± 0.0020.3160.838 ± 0.0030.275
30.858 ± 0.0060.3250.845 ± 0.0030.311
50.802 ± 0.0220.3480.792 ± 0.0060.309

HVG50: gene-wise PCC across spots, averaged over the top 50 highly variable genes.

HVG50: we select the 50 most variable genes (log1p CP10k) on the training sections only. For each gene, we compute the PCC between predicted and measured expression across the spots of each held-out section, then average over genes, held-out sections and 3 seeds. Hidden weeks: PCW 9.0 and 10.5, as two separate one-week gaps (1); PCW 9.0, 10.0 and 10.5 (3); PCW 8.0 to 11.0 (5).

TEMPUS improves over the no-memory model at every gap, most clearly on the gene-wise metric. The spot-wise gain is smaller than in extrapolation, which we expect: with observations on both sides of the target, much of the spot-wise signal is already explained by H&E.

AuthorsOpenReview ↗

Response to Reviewer guBU (2/4)

Biologically meaningful temporal change (new). On real sections, the true temporal signal cannot be separated from section and donor variation, so we also test it where the ground truth is known. Following a suggestion of Reviewer Kkea, we built a semi-synthetic benchmark from real data. We start from real Xenium sections of the developing human pancreas (6 time points, 8–17 pcw), keeping the real cells, coordinates, H&E morphology and counts, and inject a known temporal programme:

  • Niches: k-means (k = 6) on the real neighbourhood embedding.
  • Programme: each niche gets its own module of 30 genes, whose level follows a latent trajectory over time (AR(1), ρ = 0.8). It is non-monotonic, only partly predictable from the previous time point, and invisible in H&E, since morphology is unchanged.
  • Counts: Poisson-resampled from the modified expression, so realistic count noise is kept.

Because the true programme is known at every time point, we can test whether a model recovers the temporal change itself, not only per-spot accuracy. All methods use the same H&E features, targets and frozen decoder.

Semi-synthetic benchmarkTEMPUSNo memory
Extrapolation (13, 14, 17 pcw unseen), count PCC0.489 ± 0.0150.451 ± 0.012
Interpolation (12, 14 pcw unseen), count PCC0.599 ± 0.0030.584 ± 0.001
Interpolation, programme recovery (r)0.9090.428

Programme recovery: correlation between the predicted and true programme trajectory of each niche across the hidden weeks (constant prediction: 0; oracle: 0.956).

With memory, TEMPUS recovers the programme more than twice as well. Per-spot PCC changes less, because it is dominated by spatial variation that is already visible in H&E. TEMPUS therefore learns temporal change that is not present in the morphology of the target.

W2 + Q2. Explicit cross-modal alignment

We thank the reviewer for this question and tested it directly. Our reason for not enforcing alignment is that H&E does not capture all transcriptional variation, and this gap differs between sections and time points. A contrastive objective pulls both modalities into one shared space, so expression variation that H&E cannot see tends to be discarded, and this is exactly the variation TEMPUS retrieves from its temporal memory. TEMPUS instead learns its image and expression projections, and its attention over past references, from the prediction loss. Retrieval therefore selects references that help predict the target, rather than references that look alike across modalities.

We compare against two forms of explicit alignment:

  • BLEEP (Xie et al., 2023) aligns image and expression contrastively and predicts by nearest-neighbour retrieval in the aligned space.
  • TEMPUS + contrastive (new) adds a symmetric InfoNCE loss (CLIP/BLEEP style, temperature 0.07, weight 0.1) between the projected image and TERRA embeddings of each spot. Everything else is unchanged.
Count PCCTEMPUSTEMPUS + contrastiveBLEEP (matched encoder)
Human heart0.794 ± 0.0060.712 ± 0.0370.716 ± 0.100
Human pancreas0.812 ± 0.0060.802 ± 0.0020.797 ± 0.008

Adding the contrastive term lowers PCC on both datasets, most clearly on heart (0.794 to 0.712), where the temporal gain is also largest. This is consistent with the motivation above. We will add this comparison to the appendix.

AuthorsOpenReview ↗

Response to Reviewer guBU (3/4)

Seeds and significance (new). All results in the paper are mean ± sd over 3 seeds. We now add paired tests on Human heart. For each baseline, we compare TEMPUS over every (held-out time point, seed) pair, 5 test weeks × 3 seeds = 15 pairs, using a paired Wilcoxon signed-rank test and a 95% bootstrap confidence interval that resamples time points and seeds.

Baseline (count PCC)TEMPUS − baseline95% CIPairs wonWilcoxon p
TEMPUS, no memory (0.738 ± 0.008)+0.055[0.027, 0.089]15/156e-5
MERGE (0.698 ± 0.026)+0.096[0.024, 0.189]13/154e-4
BLEEP (0.671 ± 0.011)+0.123[0.071, 0.190]15/156e-5
NicheFlow† (0.626 ± 0.019)+0.168[0.117, 0.229]15/156e-5
NicheFlow+X† (0.699 ± 0.009)+0.095[0.042, 0.169]15/156e-5

*TEMPUS: 0.794 ± 0.006. † Re-run to obtain per-time-point, per-seed predictions (same split, decoder and seeds). paper numbers (*NicheFlow = 0.613 / NicheFlow+X= 0.705).

All confidence intervals exclude zero, and a Welch t-test over the per-seed means is also significant for every baseline (p ≤ 0.03). The gain over the no-memory model is significant as well (15/15 pairs), so the improvement comes from the temporal memory rather than from seed noise.

We agree that the gain on the small Visium pancreas dataset is modest. That dataset has 7 sections and a single test time point, which limits what any method can show. The gains are clearer on heart and on the Xenium pancreas (0.535 vs. 0.438 for NicheFlow+X, Table 7).

Cross-individual robustness. The split is by time point and each donor is profiled at a single age, so every test section comes from an individual never seen in training. On heart, a ridge model trained on the other individuals predicts the developmental age of a held-out individual with Spearman ρ = 0.97 (p = 0.005). The temporal signal is therefore shared across individuals (details in our reply to Reviewer Kkea, Q2).

Cross-platform robustness (new). We train TEMPUS only on Xenium sections of the developing human pancreas, with a Xenium-only memory, and test it on the held-out Visium PCW 20 section: a different study, platform, resolution and individual.

Training dataPCC ↑MSE ↓MAE ↓
Visium (paper setting)0.812 ± 0.00616.32 ± 0.070.477 ± 0.018
Xenium only (cross-platform)0.764 ± 0.02521.78 ± 0.540.416 ± 0.003

TEMPUS keeps 94% of its PCC without ever seeing the target platform, and its MAE improves.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.