acceptodds
Under review as a conference paper at ICLR 2027

Stack-JEPA: Molecular Graph Distillation for 3D Spatial Transcriptomics

20 views

Abstract

Spatial transcriptomics (ST) observes isolated two-dimensional sections, cutting anatomy and molecular neighborhoods at one arbitrary plane. Three-dimensional (3D) ST lifts that limit, but profiling a section by ST costs far more than imaging it by histology, so volumes are built by imaging every section with hematoxylin and eosin (H&E), profiling a sparse subset, and imputing the rest. Existing imputers are supervised and count-space: they regress the raw counts of one fixed gene panel from paired sections, so the objective is dominated by dropout entries, a new panel forces a refit, and accuracy is capped by an inter-section alignment computed once before the model runs. We introduce STACK-JEPA, a masked molecular graph-distillation framework for learning molecularly informed representations throughout a serial histology stack. On a profiled section, a frozen molecular teacher defines both per-location latent states and their local molecular relations. A stack student is trained to recover the held-out section’s molecular representation and relation graph from a profiled context section, the target H&E image, in-plane coordinates, and tissue depth. Frozen TERRA and UNI2-h encoders feed a transformer with 3D rotary position embeddings, while node- and relation-level distillation transfer molecular structure without placing the genepanel dimension in the backbone. After pretraining, the backbone is frozen and only a lightweight panel-specific count decoder is fitted. The same learned descriptors can replace raw expression in partial fused Gromov-Wasserstein transport, including when the target section has no measured transcriptome. Across multipe tasks, imputation on public 3D ST benchmarks and alignment under the PASTE2 subslice protocol, STACK-JEPA outperforms the supervised and generative state of the art, on imputation in both per-cell and per-gene correlation, and it aligns sections for which no transcriptome was measured, which expression-based aligners cannot handle at all.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Joint Embedding Predictive Architecture, Spatial Transcriptomics, Graph Reconstruction, 3D modelling, Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rVPeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Stack-JEPA, a self-supervised framework for learning molecular representations from serial histology sections. It combines masked-section latent prediction with graph reconstruction to preserve molecular and spatial relationships. The learned representations support gene expression imputation, spatial graph reconstruction, and 3D alignment.

Critical strengths

The paper presents a well-motivated framework for learning molecular representations from serial histology sections, achieving consistent improvements in gene expression imputation across three datasets.

The learned representations support multiple downstream tasks, including expression imputation and 3D alignment, without retraining the backbone for different gene panels.

Critical weaknesses

The ablation studies do not sufficiently isolate the contributions of JEPA training and graph reconstruction. Some reported improvements are relatively small, without statistical significance testing.

The 3D alignment evaluation relies on EMA-teacher embeddings when target expression is available, rather than the student-predicted representations. This makes it difficult to isolate the contribution of masked-section prediction to alignment performance, particularly given the substantial performance drop in the unobserved setting.

Questions for authors

Can the authors provide a coordinate-distance baseline and additional ablations isolating the JEPA objective and graph loss?

3 comments

W1 response:

We gratefully thank the reviewer for their vital comment. We agree that the original manuscript did not sufficiently demonstrate the contributions of cross-section representation training and graph reconstruction loss, and that their effects should be significance tested. We have therefore added ablations and paired statistical comparisons, and will revise our claims to reflect the scope of this evidence.

(a) Cross-section representation learning and input ablations. On HER2, we compared the full model with four variants and tested each comparison using a two-sided paired Wilcoxon signed-rank test across the eight patients, with Holm correction over the four comparisons.

ModelSVG-250 PCC_geneHolm-adjusted P
UNI2-h features + count decoder (UNI, Table 1)0.343 ± 0.0980.031
Stack-JEPA w/o expression (UNI2-h only)0.364 ± 0.1170.047
Stack-JEPA w/o H&E (TERRA only)0.369 ± 0.1180.047
Stack-JEPA, PCA of counts instead of TERRA0.402 ± 0.1010.742
Stack-JEPA0.406 ± 0.096–

Stack-JEPA is significantly higher than the UNI2-h baseline and the input ablations. With UNI2-h as its only input, Stack-JEPA cross-section modelling raises PCC_gene from 0.343 to 0.364, so the student-teacher objective adds to the frozen H&E features. Without any H&E it reaches 0.369. Replacing TERRA with a PCA of the same counts lowers PCC_gene by only 0.004, against 0.037–0.063 for the other variants. Most of the gain therefore comes from Stack-JEPA cross-section modelling, and TERRA's pretraining does not improve performance significantly.

(b) Direct ablation of graph reconstruction loss. We have extended the alignment ablation in Table 4 to cover imputation and graph reconstruction on ST-data, evaluating STACK-JEPA with and without ℒ_graph. These additional experiments directly assess the contribution of graph loss across the remaining tasks. We agree that the comparison with TERRA cannot establish this contribution, since it also reflects differences in architecture and training objectives.

Task 1:

Gene panelModelPCC_gene ↑PCC_cell ↑MSE ↓
SVG250w/o ℒ_graph0.475 ± 0.0290.810 ± 0.0230.398 ± 0.057
SVG250STACK-JEPA0.468 ± 0.0270.808 ± 0.0250.400 ± 0.056
SVG500w/o ℒ_graph0.395 ± 0.0280.784 ± 0.0230.376 ± 0.048
SVG500STACK-JEPA0.390 ± 0.0270.783 ± 0.0230.376 ± 0.049

Task 2:

ModelUnobserved AUPRC ↑Observed AUPRC ↑
w/o ℒ_graph0.630 ± 0.0190.607 ± 0.015
STACK-JEPA0.630 ± 0.0160.608 ± 0.013

Task 3:

Model3D alignment LTARI ↑
w/o ℒ_graph0.151 ± 0.025
STACK-JEPA0.165 ± 0.023

Graph loss improves alignment LTARI from 0.151 to 0.165 while the improvement from graph loss in graph reconstruction and imputation is sufficiently consistent to make any conclusions. Its contribution is therefore task-dependent, with the clearest benefit in alignment. We will revise Section 4 accordingly, remove the claim that the TERRA comparison isolates graph supervision, and will include the additional ablations in the appendix.

References

[3] Kornblith S, Norouzi M, Lee H, Hinton G. Similarity of neural network representations revisited. InInternational conference on machine learning 2019 May 24 (pp. 3519-3529). PMlR.

W2 response:

We thank the reviewer for highlighting the need to clarify the contribution of masked-section prediction to alignment performance. We would like to clarify what we report in 3D alignment task (Table 3).

PASTE2 is a protocol for performing and evaluating 3D alignment when gene expression is available for serial sections. When target expression is unavailable, the PASTE2 protocol thus cannot directly perform 3D alignment, as it requires gene expression from serial sections as input (row 1). STACK-JEPA enables 3D-aware masked-section prediction even when only H&E is available for the target section. It can therefore impute target representation and enable alignment in a setting where PASTE2 would otherwise be inapplicable (row 2). The purpose of this row is to demonstrate that 3D alignment remains feasible under this challenging condition, rather than to establish a direct comparison with the observed setting.

When expression is available for the serial sections, alignment can be evaluated under the standard PASTE2 protocol (rows 3–4). In this setting, we use the STACK-JEPA encoder rather than masked-section prediction from H&E. The resulting spot embeddings implicitly capture H&E and gene expression context from adjacent sections, yielding better 3D alignment than using raw gene expression in PASTE2. Thus, the observed results assess the utility of the learned embeddings for alignment, whereas the unobserved results demonstrate how masked-section prediction extends alignment to sections without measured target expression. We will revise the text to make these distinct purposes clearer.

Q1 response:

We thank the reviewer for the constructive suggestions, for the ablations, please see our response to your first critical weakness. We agree that comparing with naive baseline, can strengthen the significance of STACK-JEPA. However, we would like to clarify that coordinate-distance baseline is defined by spatial proximity, which is the reference for graph reconstruction task (Task 2). This task evaluates whether neighbourhoods defined by embedding similarity agree with the tissue's spatial graph: candidate spot pairs within a fixed radius are ranked by embedding similarity and scored against the spatial kNN graph, which is a coordinate-distance based graph. Coordinates define this reference graph, so a coordinate-distance baseline recovers it almost exactly (AUPRC 0.97 on ST-data, 0.99 on DLPFC).

Dear Reviewer,

Thank you again for your thoughtful review of our submission. We have put considerable time and effort into addressing your comments, including additional analyses and clarifications, and have posted a detailed response.

As the discussion period ends soon, we would be grateful if you could look at our response when you have a moment and consider whether it addresses your concerns. If so, we would kindly ask you to consider updating your assessment.

We are happy to answer any further questions or provide additional clarification.

Best regards, The Authors

Reviewer_KG45OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes STACK-JEPA, a self-supervised method for serial-section spatial transcriptomics in which only a sparse subset of sections is profiled but every section has H&E. Each spot is tokenized by fusing frozen TERRA (expression) and UNI2-h (histology) embeddings. Following the V-JEPA 2.1 recipe, a transformer with 3D rotary embeddings predicts the EMA-teacher representation of a fully masked target section from one profiled context section, the target H&E and the target coordinates. An auxiliary graph loss trains a head to reproduce teacher cosine similarities between adjacent spots. After pretraining, the backbone is frozen, and an NB count decoder is fitted per gene panel. The model is evaluated on three public datasets (HER2+, ST-data, DLPFC) for expression imputation, neighbor-graph recovery, and PASTE2-style alignment, including alignment of sections with no measured transcriptome.

Critical strengths
  • The problem is practical and clearly motivated: H&E is cheap to collect on every section while ST is not, and the paper uses the depth axis rather than predicting each section in isolation.
  • Separating a frozen representation from a panel-specific decoder is a sensible design. The panel-width experiment (Fig. 3, left) shows a single backbone serving panels from 50 genes to the full 11,871 without retraining.
  • Aligning sections that have no measured transcriptome by substituting imputed embeddings into the PASTE2 cost, is a useful capability for sparsely profiled stacks.
Critical weaknesses
  • Novelty. The method largely combines existing components: the V-JEPA 2.1 recipe (3D RoPE, EMA teacher, dense loss, deep supervision) with depth substituted for time, frozen TERRA and UNI2-h encoders, an scVI decoder, and PASTE2 with a swapped cost matrix.
    • TERRA already performs latent prediction over spatial neighborhoods, and the graph loss is a form of relational knowledge distillation that is not credited.
    • The problem setting and all three datasets are those of ASIGN and HoloTea.
    • No related-work section would let the reader judge what is new beyond the combination.
  • The modality ablation undercuts the central claim. In Table 8, the w/o-expression variant has no molecular input, yet it reaches 0.364 - PCCgene on HER2+ SVG-250. That already exceeds STFlow (0.359) and ASIGN (0.201) and approaches HoloTea (0.384). This suggests a large share of the reported gain comes from the readout and scoring pipeline rather than from 3D molecular distillation. The authors should separate these contributions.
  • A claimed contribution is absent from the method. Contribution 3 lists a fused Gromov-Wasserstein objective over the relational structure of the two representations. FGW appears only at inference for Task 3, and no such training objective is described in Section 2. The Task 2 random baseline contradicts the stated protocol. Appendix B constructs candidate sets with a positive rate of 1/6, so random AUPRC should be close to 0.17. Table 2 reports 0.37–0.39. Either the candidate construction differs from its description or the table is in error, and every Task 2 comparison depends on this.
  • The graph-loss effect on Task 2 is not ablated. Section 4 attributes the gap between the TERRA-cosine and STACK-JEPA rows of Table 2 to the graph loss. That gap also contains all of JEPA training and the H&E fusion. A without-graph-loss row for Task 2 is needed, as is already provided for alignment in Table 4.
  • Possible use of target data in gene selection. SVG panels are ranked by Moran’s I averaged across all specimens, which appears to include the held-out target sections. This affects all methods equally, but the authors should confirm the protocol or recompute the panels from training specimens only.
Questions for authors

Please see weaknesses.

4 comments

W1 response:

We are very grateful for the reviewer’s thorough review of our work. We respectfully disagree with their concern about novelty. Whilst it is true that the individual components of Stack-JEPA are established, we argue that our novelty, especially given that this is a scientific application track manuscript, is curating this specific combination of components that allows us to make three fundamental contributions to the field of 3D spatial transcriptomics:

- Sparse stack completion as masked latent prediction along depth. Prior 3D imputers, such as ASIGN and Holotea, are supervised in expression space across the serial stack; we instead mask whole sections, treating depth as V-JEPA’s time axis. We then predict the section’s latent state from a profiled section at another depth, the target H&E, and 3D position.

- Count regression-free training and gene-panel flexibility. Stack-JEPA’s latent embedding space allows for any gene panel to be cheaply fitted, requiring only the count decoder to be retrained. By comparison, prior imputation methods, such as STFlow, ASIGN and Holotea, require retraining for each panel individually.

- Alignment of sections with no measured transcriptome. Because PASTE2 requires measured expression in both sections, aligning an unprofiled section means substituting another feature cost, such as expression imputed by another method. STACK-JEPA instead provides a single representation for both profiled and unprofiled sections

(a) TERRA and relational knowledge distillation. To clarify the differences between TERRA and Stack-JEPA:

  • TERRA predicts latents within a single profiled section based on in-plane neighboring cells; Stack-JEPA predicts latents of unprofiled (or profiled) sections, across depth, based on H&E, position and a separate profiled section.
  • TERRA masks genes of in-plane neighbors, to predict gene embeddings; Stack-JEPA masks whole sections, to predict the spot-level embeddings of a whole section.

Additionally, we are grateful for the reviewer’s perceptive comment around relational knowledge distillation, which we will cite in the revised manuscript [4]. It should be noted that they differ in three ways:

  • The student never sees the target section’s expression from which the teacher’s relations are computed.
  • Relations are restricted to fixed spatial-neighbor edges.
  • Relations are read out through a message-passing head rather than matched directly between embeddings.

(b) Problem setting and datasets. We share the same problem setting and datasets of ASIGN and HoloTea by design:

  • Imputing unprofiled sections from H&E in a sparsely profiled serial stack is the central practical problem of 3D ST, due to the expensive and time consuming nature of ST. Through Stack-JEPA, we contribute a novel solution with new functionalities to this established problem.
  • The use of the same datasets is deliberate to allow for the comparison and reproduction of published results from prior methods. Our literature survey did not identify any other suitable serial-section ST datasets with H&E and multiple specimens for cross-validation.

(c) Related work. To emphasize the novelty of Stack-JEPA we highlighted in this response, we strongly agree with the reviewer and a related work section will be added to the revised manuscript.

References

[4] Park W, Kim D, Lu Y, Cho M. Relational knowledge distillation. In2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2019 Jun 15 (pp. 3962-3971). IEEE.

W2 response:

We thank the reviewer for thoughtful analysis. To address this, we read out each set of frozen features with the same count decoder, as well as with a linear readout.

HER2, SVG-250 PCC_geneHER2
UNI2-h features, count decoder0.343
Stack-JEPA, UNI2-h only0.364
Stack-JEPA, no UNI2-h0.369
Stack-JEPA, linear probe0.399
Stack-JEPA0.406

Firstly, the readout does not explain the gain, where the count decoder adds only 0.007 on top of its linear probe counterpart. With the count decoder, the UNI2-h embedding reaches only 0.343, below STFlow (0.359) and Holotea (0.384), and a linear readout of Stack-JEPA (0.399) already exceeds every HER2 baseline in Table 1. Secondly, the gain comes from 3D pretraining. Cross-section training on H&E alone lifts UNI2-h features from 0.343 to 0.364 on every fold, and the molecular input adds a further 0.042. Without any H&E, 3D molecular distillation alone reaches 0.369, already above STFlow and ASIGN. Thirdly, the comparison is like for like, since HoloTea and STFlow also read UNI H&E features and model counts with a zero-inflated negative-binomial prior. Fourthly, the scoring space does not explain the gain. As noted in W1, we use the same datasets as prior methods so that published results can be reproduced. Under our shared scoring, the baselines reproduce their published values.

W3 response:

We thank the reviewer for pointing out these inconsistencies. We address the concerns respectively.

(a) Role of FGW. We acknowledge that our current wording is not sufficiently clear. We would like to clarify that fused Gromov–Wasserstein is used for post-hoc alignment of the learned embeddings under the PASTE2 protocol in Task 3. It is not part of the STACK-JEPA training objective. We will revise contribution 3 to refer to “FGW-based alignment” and clarify that this alignment is post-hoc alignment after model training.

(b) Random AUPRC and candidate construction. Thank you for catching the discrepancy between Appendix B and Table 2. The stated positive rate of 1/6 assumes five negative pairs per positive pair. However, the candidate-radius restriction limits the available non-edges, resulting in fewer than five negatives per positive. The realized positive fractions are therefore 0.39, 0.39, and 0.37 across the three datasets, consistent with the reported random AUPRC values. The implemented edge cap is 2 grid pitches, whereas Appendix B states 2.5. We will correct Appendix B to accurately describe the implemented candidate construction and distance threshold.

W4 response:

Our original submission ablated graph loss for 3D alignment (Table 4), but not for Task 2. We agree that the comparison between TERRA-cosine and STACK-JEPA does not isolate graph supervision, as it also includes differences in JEPA training and H&E fusion. To address this, we sought to characterise the effect of the graph reconstruction loss and conduct direct ablations for graph reconstruction on ST-data.

Comparing embeddings learned with and without graph loss reveals little change in observed representations, but more pronounced differences in pairwise similarities and local neighbourhoods for unobserved representations:

Representation similarity metricMeaningObservedUnobserved
Linear CKA [3]Overall representation structure similarity0.456 ± 0.1480.289 ± 0.008
Pairwise cosine similarity Spearman correlationAgreement in similarity rankings across spot pairs0.461 ± 0.1560.341 ± 0.036
kNN overlap@6Fraction of six nearest neighbours retained0.460 ± 0.0840.267 ± 0.057

However, these differences in embedding structure do not translate into higher Task 2 AUPRC. The additional ablations provide inconclusive evidence of a benefit from graph loss for imputation and graph reconstruction under the current evaluation protocols:

Gene panelModelPCC_gene ↑PCC_cell ↑MSE ↓
SVG250w/o ℒ_graph0.475 ± 0.0290.810 ± 0.0230.398 ± 0.057
SVG250STACK-JEPA0.468 ± 0.0270.808 ± 0.0250.400 ± 0.056
SVG500w/o ℒ_graph0.395 ± 0.0280.784 ± 0.0230.376 ± 0.048
SVG500STACK-JEPA0.390 ± 0.0270.783 ± 0.0230.376 ± 0.049
ModelUnobserved AUPRC ↑Observed AUPRC ↑
w/o ℒ_graph0.630 ± 0.0190.607 ± 0.015
STACK-JEPA0.630 ± 0.0160.608 ± 0.013

The alignment gains in Table 4 nevertheless suggest that graph supervision may be useful for a different aspect of the representation: maintaining spatial relationships when establishing correspondences between sections. We investigated this possibility by assigning each source spot to its most similar embedding in the adjacent section, then assessing how those assignments alter the spacing of neighbouring source spots (Appendix Figure 6). The visualization marks reduced distance distortion in blue and increased distortion in red. The blue areas surrounding the yellow-starred example demonstrate local improvements in spatial preservation, suggesting that graph loss helps maintain the original spacing between neighbouring spots during embedding-based matching across sections. We also examined whether the alignment benefit persists under coordinate perturbations. For fold 1 of ST-data, each section was independently rotated by up to ±30° and translated by up to ±400 µm. Training and evaluation used these perturbed coordinates, while alignment quality was assessed against the measured section positions:

ModelAlignment: unobserved LTARI ↑Imputation: PCC_gene (SVG250) ↑
w/o ℒ_graph0.3460.446
STACK-JEPA0.3800.449

In this experiment, retaining graph loss improves alignment, whereas imputation changes only marginally. Our evidence thus points to a contribution that depends on the downstream task, with stronger support for 3D alignment than for within-section graph reconstruction. We will correct the attribution in Section 4, add the requested Task 2 ablation row, and document the supplementary analyses in the appendix.

References

[3] Kornblith S, Norouzi M, Lee H, Hinton G. Similarity of neural network representations revisited. InInternational conference on machine learning 2019 May 24 (pp. 3519-3529). PMlR.

W5 response:

Gene panels. We gratefully thank the reviewer for noticing this. In Tables 1, 7 and 8 of the original submission, the scored gene panels were ranked over all sections. We now rank every panel per fold on training sections only, then retrain (other methods) or refit (Stack-JEPA) every method of Tables 1, 7 and 8 on them, and report these panels in the revision. They keep 93–97% of the SVG genes and 95–100% of the HEG-250 genes, with Stack-JEPA keeping the highest PCC_gene and PCC_cell in all nine dataset-panel settings, albeit where the HER2 pcc_gene score is now closely followed by HoloTea. The revised manuscript will contain the fully updated Tables 1, 7 and 8.

SVG-250 PCC_gene

MethodHER2ST-dataDLPFC
Overlap0.170 ± 0.1360.292 ± 0.0520.085 ± 0.086
UNI2-h0.324 ± 0.1070.379 ± 0.0330.233 ± 0.075
ASIGN0.202 ± 0.1980.418 ± 0.0870.065 ± 0.066
STFlow0.346 ± 0.1180.454 ± 0.0290.217 ± 0.118
HoloTea0.402 ± 0.0890.459 ± 0.0280.215 ± 0.130
Stack-JEPA0.402 ± 0.0980.470 ± 0.0280.262 ± 0.082

SVG-500 PCC_gene

MethodHER2ST-dataDLPFC
Overlap0.118 ± 0.0970.211 ± 0.0480.051 ± 0.056
UNI2-h0.256 ± 0.0790.315 ± 0.0340.183 ± 0.057
ASIGN0.152 ± 0.1520.324 ± 0.0910.040 ± 0.046
STFlow0.272 ± 0.1070.363 ± 0.0510.166 ± 0.101
HoloTea0.306 ± 0.0780.345 ± 0.1260.174 ± 0.086
Stack-JEPA0.327 ± 0.0800.391 ± 0.0290.203 ± 0.066

Dear Reviewer,

Thank you again for your thoughtful review of our submission. We have put considerable time and effort into addressing your comments, including additional analyses and clarifications, and have posted a detailed response.

As the discussion period ends soon, we would be grateful if you could look at our response when you have a moment and consider whether it addresses your concerns. If so, we would kindly ask you to consider updating your assessment.

We are happy to answer any further questions or provide additional clarification.

Best regards, The Authors

Reviewer_RM7UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces STACK-JEPA method for imputing ST for tissue sections that have only H&E images, using the ST and H&E of a context slice and the H&E of the target slice within the same specimen. Two frozen pretrained encoders, TERRA and UNI2-h, encodes molecular and morphological features into one token for each spot of ST. The distillation network is pre-trained by predicting teacher’s latent representations of the masked target ST and by reconstructing a molecular graph that recovers the similarity between neighboring spots’ embeddings. After pretraining, the learned representations are used for three tasks: gene-expression imputation with an scVI-style decoder, spatial graph reconstruction, and 3D alignment using PASTE2 with learned embeddings. The authors evaluate STACK-JEPA on three datasets and claimed improvements over existing methods.

Critical strengths
  1. The paper addresses an important problem of reconstructing 3D ST data from sparsely profiled tissue sections where both gene expression and H&E images are available.
  2. STACK-JEPA adapts V-JEPA approach to learn latent representations rather than directly predicting gene expression. A self-supervised backbone can support three different tasks, and enable 3D alignment where target expression was not measured.
Critical weaknesses
  1. The student-teacher framework contribution is unclear. STACK-JEPA replies on two pretrained encoders, TERRA and UNI2-h, it is not clear how much the student teacher network contributed to the performance improvement. The UNI baseline in table 1 nearly matches STACK-JEPA on DLPFC. These results raise questions about whether STACK-JEPA provides significant improvements over the baselines.
  2. The benchmarking is insufficient. In table 3’s target expression unobserved section, STACK-JEPA’s is the only entry with numerical results. The authors should compare against PASTE2 with UNI2-h features or PASTE2 on imputed expression by HoloTea or STFlow. The authors should also compare with recently developed spatial foundation models, such as NicheFlow, where applicable. Moreover, representation generalizability across datasets remains unclear, as the current evaluation is mainly conducted within the same datasets.
Questions for authors
  1. Since the encoder and predictor apply full attention over whole section pairs, I expect the current method’s computational cost to grow quadratically as the number of spots increase per slice. What are the training and inference time and memory requirements? And what is the intended scale of the method? How well would the method scale to larger sections or higher resolution ST datasets?
  2. Are the baseline methods also trained on the held-in sections of the held out specimen, as STACK-JEPA is?
  3. In table 2, why does STACK-JEPA achieve higher AUPRC with unobserved target expression on ST-data and DLPFC datasets? Isn’t graph reconstruction more difficult when target expression is unobserved? More importantly, since the evaluation measures spatial adjacency, it is unclear whether the improvements reflect meaningful molecular information or simply spatial structure already available from the coordinates. The authors should have a direct evaluation of molecular similarity preservation.
  4. In practice, how are prediction locations defined for genuinely H&E-only sections? Does STACK-JEPA require a predefined spatial grid or additional registration across sections?
  5. Table 4 evaluates the effect of graph loss on 3D alignment, its contribution to the other two tasks remains unclear. In Section 4, the authors claim that comparing STACK-JEPA with TERRA embeddings isolates the effect of graph supervision. However, this comparison also involves differences in architecture and training objectives. A direct ablation with and without the graph loss across all three tasks would better support this claim.
5 comments

W1 response:

We thank the reviewer for their insightful comment. We agree that the original manuscript did not properly demonstrate that the student-teacher training was responsible for the performance improvement of Stack-JEPA over the baselines and benchmarks. To address this, we have designed a set of ablation experiments to exemplify this:

SVG-250 PCC_geneHER2
UNI2-h + same count decoder0.343 ± 0.098
Stack-JEPA without expression (UNI2-h only)0.364 ± 0.117
Stack-JEPA without H&E (no UNI2-h)0.369 ± 0.118
Stack-JEPA, PCA of counts instead of TERRA0.402 ± 0.101
Stack-JEPA0.406 ± 0.096

Firstly, trained with the same UNI2-h input and count decoder, the inclusion of the Stack-JEPA cross-section modelling increased PCC_gene from 0.343 to 0.364. Secondly, without any H&E Stack-JEPA still reaches 0.369, above UNI2-h features on every fold, so UNI2-h alone does not explain the gain. Thirdly, replacing TERRA with a PCA of the counts changes PCC_gene slightly by 0.004, so the gain is not fully attributable to TERRA's pretraining either. We also agree that the DLPFC margin of Stack-JEPA over UNI2-h is small, with a margin of 0.013 (SVG-250 PCC_gene). This can be explained by the inherent characteristics of the dataset; DLPFC favors H&E most, since its cortical layers are broad bands that morphology can resolve cleanly. Therefore, cross-section context from Stack-JEPA provided little improvement in imputation performance when tissue architecture is clearly resolvable from H&E, which we will add to the limitations.

W2 response:

We sincerely thank the reviewer for their constructive comments, which help us better articulate the significance of STACK-JEPA. We agree that extensive benchmarking with more baselines and more dataset will strengthen our claim. We address each point in turn below.

(a) Task 3 alignment benchmarking. For the 3D alignment task (Table 3), we added PASTE2 baselines using gene expression imputed from H&E by HoloTea in the unobserved setting. We’d like to clarify that the PASTE2 evaluation protocol requires gene expression as input. Since UNI2-h produces histology embeddings and does not itself impute gene expression without needing to fit an additional count decoder on top, using UNI2-h embeddings falls out-of-scope from benchmarking.

MethodST-dataDLPFC
Target expression unobserved
PASTE2 on HoloTea-imputed expression0.102 ± 0.0200.176 ± 0.055
STACK-JEPA0.145 ± 0.0270.204 ± 0.078
Target expression observed
PASTE20.148 ± 0.0230.367 ± 0.029
STACK-JEPA0.165 ± 0.0230.408 ± 0.039

(b) Foundation model benchmarking. The reviewer has made an excellent suggestion; we have added NicheFlow [1] as a baseline to Table 1. Since NicheFlow models how niches evolve between time points rather than predicting expression at given locations from histology, so we adapted it towards the serial sections whilst retaining the original training objective. Each target section is generated from its nearest provided section, and each target location takes the expression of the nearest generated cell, mapped back to genes by inverse PCA. NicheFlow uses no histology and models temporal niche trajectories, so its results represent expression-only transfer.

SVG-250 PCC_geneHER2+ST-dataDLPFC
NicheFlow (adapted)0.069 ± 0.0950.294 ± 0.0550.104 ± 0.040
HoloTea0.384 ± 0.0960.398 ± 0.1490.235 ± 0.101
Stack-JEPA0.406 ± 0.0960.468 ± 0.0270.257 ± 0.083

W2 response continued:

Representation generalisability. We strongly agree that generalizability across datasets is vital for demonstrating robustness. We first looked for additional benchmark datasets, but our survey found no others that are suitable (satisfying H&E and ST on multiple serial sections per specimen, and multiple specimens for cross-validation). For example, OpenST [2] contains only a single sample, which rules out the cross-validation splits needed for a fair evaluation. HER2, ST-data and DLPFC therefore remain the only widely adopted benchmarks that allow direct comparison with competing methods.

To assess generalization within these constraints, we designed a cross-dataset transfer experiment. ST-data and HER2 both profile breast cancer, so this setup tests whether the representation learned on one dataset carries over to another dataset of the same tissue type. We froze each of the 4 backbones trained on ST-data, fitted only the count decoder on HER2 fold 1, and evaluated on the remaining unseen HER2 sections in fold 1, which the backbones were never trained on. With the same count decoder, every transferred backbone outperformed UNI2-h.

ST-data → HER2 Fold 1SVG-250 PCC_gene
UNI2-h0.138
Stack-JEPA, transferred backbone (mean ± s.d. over 4 backbones)0.179 ± 0.004
Stack-JEPA, backbone trained on destination0.207

Evaluating STACK-JEPA on new public benchmarks as they become available would further strengthen the paper, and we consider this an important direction for future work.

References [1] Sakalyan K, Palma A, Guerranti F, Theis F, Günnemann S. Modeling microenvironment trajectories on spatial transcriptomics with nicheflow. Advances in Neural Information Processing Systems. 2026 Apr 23;38:103328-71. [2] Schott M, León-Periñán D, Splendiani E, Strenger L, Licha JR, Pentimalli TM, Schallenberg S, Alles J, Tagliaferro SS, Boltengagen A, Ehrig S. Open-ST: High-resolution spatial transcriptomics in 3D. Cell. 2024 Jul 25;187(15):3953-72.

Q1 response:

The reviewer’s insight is correct; attention is dense over each section pair, so the cost grows quadratically with spots per pair.

MeasuredHER2ST-dataDLPFC
Spots per section166–706296–7063,320–4,641
Pretraining per fold (4 × H200)2.4–2.5 h2.4–2.5 h9.1–10.3 h
Peak GPU memory per card, pretraining43–45 GiB43–53 GiB65–70 GiB
Peak host RAM, pretraining27–32 GiB27–34 GiB44–51 GiB
Inference per held-out section (1 × L40S)0.04–0.10 s0.06–0.12 s0.09–0.20 s
Peak GPU memory, inference0.31–0.36 GiB0.33–0.36 GiB0.49–0.55 GiB
Peak host RAM, inference≤ 1.9 GiB≤ 1.9 GiB≤ 1.9 GiB

The method targets whole tissue blocks at spot resolution. Every section has H&E, a few have ST, and Stack-JEPA predicts each remaining section whole from its whole-slide H&E. For example, a Visium section holds at most 4,992 spots and at spot resolution, the structures of interest are section-scale, so whole-section attention is tractable and apt. At cellular resolution the relevant structure is finer, so attention would run within local windows across adjacent sections, keeping cost linear in section area. Cell-resolution serial-section ST datasets with multiple samples remain scarce, so we list this as a limitation.

Q2 response:

We thank the reviewer for raising this important question that underpins the validity of the benchmarking. Yes, every method trains on the same sections and is scored on the same target sections. For example, in fold 1 of the HER2 dataset:

TrainingProvided sections of held-out specimensTarget sections
B, C, D, F, G, HA1, A2, E1, E2A3–A6, E3

(Each letter corresponds to a given specimen and each number refers to the section position. Specimens A and E have 6 sections and 3 sections, respectively.)

Q3 response:

We thank the reviewer for constructive criticism. Your comment raises two main points:

(a) AUPRC observed vs unobserved. Let us clarify the “observed” vs. “unobserved” settings. In an “unobserved” setting, the model cannot see the target sections’ gene expression. Instead, it uses the target section’s tissue image and spot coordinates, together with gene expression from neighboring sections along the z-axis, to predict the target section’s spot-level kNN graph. In the “observed” setting, the model can also see the target section’s gene expression and use it directly to predict the spot kNN graph. As the reviewer points out, the observed setting explicitly uses each spot’s measured expression, which would generally be expected to facilitate spot kNN graph reconstruction. Since we observed slightly higher performance in the unobserved setting on some datasets, we hypothesized that this might reflect greater spatial smoothness in the predicted representations. Encodings of measured expression retain spot-level variation, including biological heterogeneity and measurement noise, whereas predictions from an adjacent section may smooth over some of this variation. To examine this hypothesis, we compared the spatial similarity patterns of observed and predicted representations on ST-data and DLPFC:

DatasetNeighbour cosine, observedNeighbour cosine, unobservedDistance–similarity correlation, observedDistance–similarity correlation, unobserved
ST-data0.600.79−0.40−0.46
DLPFC0.770.93−0.21−0.36

These results support our hypothesis that predicted representations are spatially smoother, which may contribute to greater agreement with the spatial kNN graph.

(b) Evaluation of molecular similarity preservation. We agree that the spatial kNN graph reconstruction task (Task 2) primarily evaluates the preservation of spatial organization. This provides a fundamental assessment of whether neighborhoods defined by embedding similarity reflect the tissue's spatial structure. We agree that this evaluation alone does not establish that molecular similarities are preserved. To evaluate molecular similarity directly, we now score Task 2 by edge Spearman. For each target section, we take the true spatial neighbor pairs and compute the Spearman correlation between two similarities: the cosine similarity of the embeddings and that of the measured SVG-250 expression. Due to the spot array-based nature of the data, spatial neighbors are equally far apart, so coordinates cannot rank them. As references, we also score two representations that see the target expression: Stack-JEPA’s encodings of the measured sections and TERRA embeddings of the measured expression.

Edge Spearman (SVG-250) ↑HER2+
Coordinates0.060 ± 0.042
UNI2-h0.289 ± 0.082
Stack-JEPA0.308 ± 0.116
Stack-JEPA, measured target (reference)0.324 ± 0.116
TERRA, measured expression (reference)0.392 ± 0.070

Coordinates score near zero, so edge Spearman separates molecular similarity from spatial layout. Stack-JEPA’s states keep nearly all the molecular similarity of its encodings of the measured sections and score above UNI2-h features of the target’s own H&E. Decoding these states into expression raises their edge Spearman to 0.371, the highest of all methods.

Q4 response:

Here, the reviewer raises a fundamental question, which we will clarify within the revised manuscript. The model takes any set of query locations, such as a spot grid over the tissue mask or segmented nuclei. In the benchmark the queries are the measured spot positions; sections were not pre-registered prior to model training or inference.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear reviewers and AC,

We thank all reviewers for their time, expertise, and constructive feedback, and for recognising STACK-JEPA’s molecular representations across serial sections, their applicability across tasks and gene panels, improved expression imputation, and ability to align sections without measured transcriptomes. The reviewers raised several points:

  • Novelty and component contributions What STACK-JEPA adds beyond existing methods, and whether JEPA training and graph supervision provide benefits beyond pretrained encoders and the count decoder.

  • Molecular information preservation. Whether graph reconstruction demonstrates molecular similarity preservation beyond spatial proximity and smoothness.

  • Alignment evaluation. Whether appropriate baselines and comparisons between observed and predicted representations establish the contribution of masked-section prediction.

In response, we have carefully considered and addressed each comment and concern point by point; conducting extensive new experiments to strengthen the empirical support for our findings and clarifying the novelty of our contributions:

  • Through controlled modality and encoder ablations, matched-decoder comparisons, linear probing, and paired statistical tests, we provide stronger evidence that STACK-JEPA’s gains arise from cross-section representation learning beyond its pretrained features and decoder.

  • We complemented spatial graph evaluation with coordinate-distance baselines, smoothness analyses, and direct molecular-similarity measurements against observed expression. These experiments provide direct evidence that STACK-JEPA’s predicted representations capture molecular information and preserve local spatial neighbourhood structure.

  • We expanded benchmarking with NicheFlow for imputation and PASTE2 on HoloTea-imputed expression for alignment. When target expression is unmeasured, Stack-JEPA achieves higher alignment accuracy, strengthening the evidence for the practical value of its predicted molecular representations.

  • Ablations of the graph reconstruction loss across all three tasks identify a clear task-specific contribution to 3D alignment, including under coordinate perturbations.

Together, these analyses strengthen the evidence for STACK-JEPA’s performance advantages across all three tasks, while clarifying the sources of its gains and the task-specific role of graph supervision. We have incorporated the additional experiments and analyses into the revised manuscript, clarified the framework’s novelty and limitations, refined our claims to reflect the task-specific contributions of its components, and resolved the gene panel selection issue.

Our point-by-point responses provide detailed results and clarifications. We welcome any further questions or suggestions from the reviewers and AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.