acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing spatial prediction in histology representations

Abstract

Predicting the cellular and molecular state of surrounding tissue offers a test of what histology representations encode beyond a local image. A spatial prediction score, however, can rise for reasons other than prediction of unseen tissue: the comparison can hide the baseline, the target can favor one encoder, and the input can see part of the target. We develop a diagnostic protocol that isolates each effect with matched evaluations across thirteen TCGA cancers and colorectal (Xenium) and renal (Visium) spatial-omics cohorts. Retrained TRIPLEX and MERGE show 74–92% smaller context gains on raw expression than on targets averaged over neighboring tissue already visible to the model, and neither method beats the training mean on raw renal targets. With six frozen encoders, an advantage in predicting nearby over distant tissue coexists with both predictions below the training mean, and the UNI2-H versus ResNet50 gap shrinks by 77–78% on shared measured targets while pathology encoders keep their ranking. Notably, images of the target region predict it where local crops cannot, and two slide encoders show no gain that clears a prespecified criterion once the target is withheld. Local UNI2-H features still improve nearby cell-composition prediction beyond a reference from the crop's measured gene profile, and a case study of a spatially supervised encoder shows that its neighborhood gain depends on the target, readout, and baseline. We distill these diagnostics into three reporting requirements: absolute scores for both compared targets, shared measured targets for encoder comparisons, and a declaration of the tissue each input and target can see.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Computational pathology, spatial transcriptomics, representation evaluation, tissue context

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qHYYOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines the reliability of spatial prediction evaluations in histology representations. The authors propose a diagnostic framework focusing on three issues: misleading relative performance gains, encoder-specific prediction targets, and overlap between visible tissue and prediction targets. Through experiments across multiple cancer cohorts and pathology encoders, the study shows that spatial prediction gains are sensitive to target construction, evaluation metrics, and tissue accessibility. The authors recommend more rigorous evaluation protocols to better assess spatial information in histology representations.

Critical strengths
  1. The paper provides a systematic evaluation of spatial prediction in histology representations, examining multiple factors that may influence the interpretation of prediction performance.
  2. The experiments are relatively comprehensive, covering multiple cancer cohorts, pathology foundation models, and spatial transcriptomics datasets. The analyses of target smoothing, shared prediction targets, and tissue accessibility provide useful empirical insights into existing evaluation practices.
Critical weaknesses
  1. Insufficient justification of unseen-tissue prediction. The paper does not clearly establish why predicting unseen surrounding tissue is a meaningful measure of spatial representation quality, particularly given the biological heterogeneity of tumor tissues.
  2. Mismatch with the objectives of prior work. Methods such as TRIPLEX and MERGE aim to improve gene expression prediction using visible spatial context, rather than predicting unseen tissue. The relevance of the proposed evaluation to their original claims is insufficiently justified.
  3. Questionable interpretation of reduced context gains. Spatial smoothing is a reasonable strategy for handling noisy spatial transcriptomics data. The reported 74–92% reduction in context gains may partly reflect changes in prediction targets and task difficulty, rather than misleading improvements in prior methods.
  4. Limited significance of the empirical findings. Several findings largely confirm expected effects, such as improved prediction when the target tissue is visible and changes in encoder comparisons when prediction targets differ. The additional scientific insights beyond these established principles remain unclear.
Questions for authors
  1. Why is predicting unseen surrounding tissue an important way to evaluate pathology representations? Could the authors explain what this task tells us about a model's capabilities and how it relates to practical pathology applications?
  2. TRIPLEX and MERGE mainly use visible neighboring tissue to improve gene expression prediction. Could the authors clarify which claims of these methods are challenged by evaluating prediction of unseen tissue?
  3. Spatial smoothing is a reasonable part of the original prediction tasks. Could the authors explain why the reduced gains on raw expression should be considered evidence of potentially misleading improvements, rather than simply reflecting a different prediction target?
  4. Several results seem expected, such as better prediction when the target tissue is visible. Could the authors explain what new insights their experiments provide beyond these observations, or give a concrete example where the proposed evaluation changes the interpretation or selection of a pathology model?
8 comments
AuthorsOpenReview ↗

Response to Reviewer qHYY (1/4): why withhold the target tissue (W1/Q1)

Thank you for the review. We answer the four questions in turn.

Withholding is a control, not a proposed measure of representation quality. A spatial gain can come from recognizing the tissue in the input, from using additional tissue the model can see, or from inferring tissue outside the input. The first two are legitimate, and for prediction from an available slide they are the right design. Only the third requires the target to be withheld. It is also the claim made in practice when a gain is attributed to spatial context, or when an encoder is described as encoding its neighborhood. Withholding the target is how that claim is tested; failing the test does not mean a model lacks value.

Evidence that separates the two. On the same 320–640 µm ring targets, both local crop widths score below the training mean for every encoder, whereas an MLP on tiles inside the ring reaches R2R^2 = 0.26–0.31 (Section 4.4, Table S23). The target is predictable from H&E; what fails is inference beyond the input. Heterogeneity is handled by design: each true ring is compared with a count-matched far ring from the same slide under the same eligibility rules, so slide-level heterogeneity affects both arms.

Scope. The direct-access result holds with a flexible readout; with ridge, ring tiles also stay below the training mean. A positive true-minus-far contrast shows a predictive association with the local neighborhood, not a mechanism.

AuthorsOpenReview ↗

Response to Reviewer qHYY (2/4): smoothed targets and task difficulty (W3/Q3)

We agree that smoothing is a reasonable way to handle noisy spatial transcriptomics, and we do not claim that the prior improvements are incorrect or misleading; Figure 1's "can mislead" refers to how a score is read, not to those results. The question is what a smoothed-target gain can be attributed to. Switching to raw targets changes two things at once, target noise and whether the input contains tissue that enters the target, so we test them separately.

Evidence. Keeping the smoothed target fixed, we mask from TRIPLEX's neighborhood view the adjacent capture areas that enter it. The task and its difficulty are unchanged, yet the gain in mean Pearson rr falls from 0.062 to 0.014 in ccRCC and from 0.075 to 0.015 in Xenium, reductions of 77% and 80%, to about the size of the raw-target gains (0.016 and 0.011; Table S16).

Scope. Masking applies to TRIPLEX; MERGE has no neighborhood view, and its spatial-edge gain shows the same smoothed-versus-raw pattern under coordinate permutation (Table S18). Target noise explains part of the difference: after reliability normalization, the target-by-view interaction falls from 0.046 to 0.037 in ccRCC and from 0.064 to 0.053 in Xenium, with both intervals above zero (Appendix F.9). A smoothed-target gain therefore measures better prediction of neighborhood-averaged expression, part of which the model can see. It supports a claim about each spot's own expression only together with a control such as masking.

AuthorsOpenReview ↗

Response to Reviewer qHYY (3/4): which claims of TRIPLEX and MERGE are tested (W2/Q2)

This test involves no unseen tissue. Both methods were retrained with their released code, hyperparameters and inputs, visible context included; we changed only the target, from smoothed to raw spot expression, and for TRIPLEX also masked the neighbors entering the smoothed target (W3/Q3). We do not question their use of visible context or their gains on smoothed targets, which we reproduce. The claim tested is narrower: that these gains show context improves prediction of each spot's own expression.

Evidence. On raw targets, no context gain in either method or cohort has a global R2R^2 interval above zero: TRIPLEX 0.009 [−0.005, 0.021] and 0.008 [−0.008, 0.023], MERGE 0.004 [−0.006, 0.013] and 0.004 [−0.007, 0.016] for ccRCC and Xenium (Table S15). On smoothed targets, all four intervals are above zero.

Scope. Two of the three published context claims replicate and are narrowed; the TRIPLEX whole-slide gain does not replicate on our renal cohort, so we draw no conclusion about it (Table S21). The results come from retraining on our cohorts, not the original data. These results narrow the interpretation of the published gains; they do not overturn the methods on the tasks they were designed for.

Reviewer_ka76OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper mainly investigates what spatial prediction experiments reveal about histology representations. The central argument is that a good spatial prediction score can come from a few different sources. Specifically,

  1. A nearby target is predicted better than a distant target, even though both predictions are worse than a training-mean predictor.
  2. Encoders are compared using different targets, especially when each encoder predicts its own neighborhood embeddings.
  3. An expanded image input includes tissue contributing directly to the prediction target.

The paper builds a protocol around matched nearby and distant annulus. Separate probes predict the two targets from the same query representation and they evaluate on multiple frozen tile encoders, slide encoders, and a supervised SEAL checkpoint.

This is a careful rethinking/audit style paper and I believe it can be helpful to the subfield and community.

Critical strengths

1. The evaluation problem is well motivated

The distinction between predictive accuracy, neighborhood specificity, and access to the target tissue is very useful and well-motivated, because these quantities answer different questions, and the paper provides concrete examples of why reporting only one can be misleading. I totally agree with it.

This is a reasonable contribution for an empirical evaluation paper, as its value should be empirically changing how researchers interpret results.

2. The experiment comparisons are carefully designed

It is sound and reasonable to me that the paper keeps queries, patient folds, and far partners fixed within comparisons. It uses splits grouped by patient, keeps paired tumor and normal sections together, and treats patients as the units of inference. It also documents the different analysis sets instead of simply comparing results from incompatible subsets.

3. Actionable outcome

So the three reporting requirements need no new model or data and are easy for authors or readers in this subfield to adopt.

Critical weaknesses

1. (Major) Model selection is not aligned with the primary reported metric

The findings on tissue access change with the choice of readout. In Sec. 4.4, direct ring access achieves positive R2R^2 of 0.26–0.31 with MLPs, whereas ridge scores remain negative −0.170 to −0.088 and even underperform the 320µm crop for UNI2-H and Phikon-v2. Although alternative ridge model selection improves UNI2-H to R2=0.063R^2=0.063, this remains well below the MLP result of 0.308.

Similarly, while Section 4.5 reports that neither slide encoder meets the prespecified MLP-based context-gain criterion, Prov-GigaPath with ridge achieves a positive ring-withheld context gain of 0.093 and a positive change of 0.056 in true-minus-far contrast. These results show that the interpretation depends strongly on the readout, but different sections emphasize different primary readouts. The authors should present these discrepancies more prominently and qualify their conclusions about direct target access and the benefit of target-withheld slide context.

2. Pathology encoder order preserving claim is somehow strong

The abstract says that pathology encoders “keep their ranking,” the Sec. 4 heading also refers to “preserving their ordering” , but looking at Tab 9, on the shared measured target, Phikon-v2 falls below all three natural-image encoders, and ViT-B/16 exceeds both Phikon-v2 and Prov-GigaPath. These statements overstate the measured cell-composition results.

3. The gap reduction should not be interpreted (or at least soften) as self-space advantage From my understanding, the shared measured target experiment changes both whether the target is common and what is being predicted. An encoder’s neighborhood embedding, four-class cell composition, and a fifty-gene profile are different prediction tasks.

Reliability normalization addresses part of the measurement problem, but it cannot make these targets equivalent. It is pretty clearly illustrated in Table S8, as using a common embedding target can enlarge or nearly eliminate the gap, depending on the target chosen.

So I would suggest that the strongest conclusion is that encoder comparisons are target dependent, and measured biological targets can produce substantially different conclusions.

Questions for authors
  1. Figure 1 contains minor spelling errors: “Patholgy FMs” and “General Vison.”
  2. 7 analysis-set codes plus two scoring conventions, two penalty grids and two readouts make it hard to track which protocol produced which number. A single main-text table listing set, readout, grid, or score convention per result would be very helpful.
4 comments
AuthorsOpenReview ↗

Response to Reviewer ka76 (1/2): readout dependence

Thank you for a careful and constructive review. We have carefully revised the manuscript to address each of the points raised.

W1. Readout dependence. Thanks for raising this point. Each conclusion in Sections 4.4 and 4.5 now names the readout under which it holds:

  • Section 4.4: "This holds for the MLP readout only: with ridge, ring tiles score below the training mean (−0.170, −0.141, −0.088) and, for UNI2-H and Phikon-v2, below the 320 µm crop."
  • Section 4.5: "With ridge, Prov-GigaPath's ring-withheld token improves global R2R^2 by 0.093 [0.056, 0.135] and the contrast by 0.056."
  • The abstract, contribution (ii) and conclusion now qualify the direct-access result with "with a flexible readout".

The full comparison (Tables S11, S13, S23; Appendix F.4):

Comparison (analysis set)RidgeMLP
UNI2-H, 320 minus 80 µm crop, nearby ring (X-Near)0.141 [0.087, 0.196]0.230
Ring tiles, true R2R^2, UNI2-H / Phikon-v2 / ResNet50 (X-Ring)−0.170 / −0.141 / −0.0880.308 / 0.263 / 0.272
UNI2-H, ring tiles minus 320 µm crop (X-Ring)−0.129 [−0.312, 0.046]0.340 [0.180, 0.508]
Prov-GigaPath, ring withheld minus single tile (X-Context)0.093 [0.056, 0.135]0.003 [−0.177, 0.141]
TITAN, ring withheld minus single tile (X-Context)0.0330.088 [−0.176, 0.334]
Ring-withheld true R2R^2, TITAN / Prov-GigaPath (X-Context)−0.118 / −0.017−0.292 / −0.196
Conditional image gain GG, gene-profile reference (X-CropRef)0.041 [0.011, 0.074]0.048 [−0.025, 0.160]

Two conclusions hold under both readouts: visible target tissue improves prediction (row 1), and no ring-withheld input beats the training mean (row 6). Two depend on the readout: direct ring access beats the local crop only with an MLP, and Prov-GigaPath's withheld context helps only with ridge. The Prov-GigaPath case is the pattern that reporting requirement 1 is meant to catch: a real gain (+0.093) that still leaves the score below the training mean (−0.017).

Model selection and the primary metric. We agreed that the two were not aligned: ridge penalties were selected by query-pooled validation MSE, while reported scores weight patients equally. Reselecting penalties by patient-averaged validation R2R^2 on the same grid raises UNI2-H's ring-tile score from −0.170 to +0.063. This narrows the ridge–MLP gap from 0.478 to 0.245, but ridge stays well below the MLP (0.308) (Appendix F.5, Table S12). Because the alternative rule was chosen after the discrepancy was seen, we report it as a post hoc sensitivity analysis and keep the original rule as primary.

Primary readout by section. Each result has one primary readout, listed in Table S26: ridge for the encoder comparisons (4.3), the conditional image gain (Table 2) and the SEAL case study (4.6); the prespecified MLP criterion for slide context (4.5); and both readouts side by side for crop width and ring tiles (4.4). Where the two readouts disagree, the text now says so. The last row of the table above is included for completeness: the conditional image gain has an interval above zero only under ridge.

AuthorsOpenReview ↗

Response to Reviewer ka76 (2/2): encoder claims and presentation

W2. Encoder ordering. Agreed. "Keep their ranking" and "preserving their ordering" overstated the colorectal result. Revised wording:

  • Abstract (contribution (ii) uses the same terms): "the UNI2-H versus ResNet50 gap shrinks by 77–78% on shared measured targets, while the pathology-encoder advantage persists on renal gene expression and in every non-self embedding space."
  • Section 4.3 heading: "Encoder comparisons depend on the target space."

The advantage holds on renal gene expression, where every pathology encoder exceeds every natural-image encoder (contrasts 0.047–0.070 versus 0.019–0.027; cross-family win rate 79% [70%, 88%]; Tables S9 and S14). On colorectal cell composition the ranking does not hold, as you note: Phikon-v2 (0.079) falls below all three natural-image encoders (0.081–0.106), and ViT-B/16 (0.106) exceeds Phikon-v2 and Prov-GigaPath (0.099), with every interval including zero (Table S9). Section 4.3 now states only that UNI2-H keeps the largest contrast and the win rate favors pathology encoders (65% [44%, 85%]), while the other encoders overlap within patient-level uncertainty.

W3. Gap reduction is not a measure of self-space advantage. Agreed, and we adopted your conclusion. Embedding, cell-composition and gene-expression targets are different tasks, so the 77–78% reduction shows how much the comparison changes with the target; it does not measure the size of a self-space bias. Reliability normalization addresses target noise, not task differences, and the manuscript uses it only to check that the reduction is not explained by noise. Section 4.3 now reports the cross-encoder matrix directly (the UNI2-H versus ResNet50 difference is 0.303 in UNI2-H space and −0.002 [−0.010, 0.005] in ResNet50 space; Table S8) and concludes:

"Encoder comparisons are therefore target-dependent: encoder-specific targets amplify the gap, and a shared measured target narrows it while the pathology-encoder advantage remains clearest on renal gene expression."

Figure 1. The typos are corrected.

Protocol table. New main-text Table 1 lists, for each main result, the data, what is varied and what is held fixed, and the reported quantity, grouped by the three questions that now organize Section 4. New Table S26 (Appendix K) gives the analysis set, readout, penalty grid, score and primary endpoint for each result. To fit the page limit, Table 1 states the score convention in its caption and Table S26 carries the per-result readout and penalty grid. The former Table 1 is now Table 2; appendix tables S1–S25 keep their numbers.

Thank you for the detailed response. While the updated interpretation narrows the scope of the claims a little bit, it makes the paper more accurately positioned. I would like to maintain my weak accept score and raise the soundness score by 1.

Thank you for reading the revision and for raising the soundness score! Your comments made the paper's claims more accurate.

Reviewer_YeeCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This manuscript present a clear flaw in previous work (TRIPLEX/MERG). The authors demonstrate that much of the reported benefit of spatial modeling disappears when the prediction target is no longer constructed using measurements from tissue already visible to the model. The data suggests to me that there is a target–input overlap problem that the other methods just happen to be better at exploiting than other methods. Unfortunately, the manuscript is better at challenging how spatial transcriptomics prediction is evaluated than for demonstrating improved spatial prediction, at least in a meaningful way.

Critical strengths

-The strongest finding is that a large fraction of the apparent benefit of spatial context in TRIPLEX and MERGE is attributable to target smoothing and access to tissue contributing to the prediction target. The gains attributable to spatial context decrease by approximately 74–92% when smoothed targets are replaced by raw expression.

  • The use of patient-level splits, training-mean baselines, shared biological targets, matched spatial controls, and target-withholding experiments provides a thoughtful framework for evaluating spatial prediction claims. -Emphasis on the distinction between predicting a measurable biological quantity and predicting an encoder-defined representation is particularly important.
Critical weaknesses

-I find understanding what the author(s) are thinking to be very challenging. Although the underlying argument is important, the manuscript is difficult to follow. The numerous experimental configurations, target definitions, contrasts, and evaluation metrics make it challenging to understand which findings are fundamental and which are supporting analyses. A restructuring of the presentation would make the manuscript much more like to be acceptable at a similar conference. -The authors do not establish what magnitude of improvement would be biologically consequential despite pointing out the differences observed by different experiments.

The usual weaknesses of spatial transcriptomics image analysis:

  1. Small cohorts, evaluation of tens of thousands of spatial locations, those observations are not statistically independent within the small cohorts.
  2. predicting distinct morphology is not equivalent to predicting gene expression.
  3. The poor absolute performance leads one to wonder if analyses like these have a meaningful destination even if methods were perfect.
Questions for authors

Is a delta of 0.05 in correlation compelling for purposes of believing a model is predicting this complex target? Could the authors more clearly distinguish the primary methodological findings from secondary benchmarking observations and articulate how their results should change the design and evaluation of future spatial transcriptomics prediction methods?

6 comments
AuthorsOpenReview ↗

Response to Reviewer YeeC (1/3): structure and effect size

Thank you for the review. The paper's primary finding is the one you identify: most of the TRIPLEX and MERGE context gain depends on targets that average over tissue the model already sees. The paper does not ask whether methods must predict unseen tissue; it asks what a spatial gain can be attributed to. The revision makes this structure explicit and states each claim only where the evidence supports it.

Primary versus supporting findings. Section 4 now opens with three questions, one per reporting requirement:

  • Q1. Is a gain above the training-mean baseline? (Sections 4.1, 4.2)
  • Q2. Does an encoder advantage hold on a shared measured target? (Section 4.3)
  • Q3. Did the input see the target? (Section 4.1, extended in Sections 4.4 and 4.5)

New Table 1 maps each result to its data and to what is varied and held fixed. The conditional image gain (Table 2) and the SEAL case study (Section 4.6) are labeled as supporting analyses; no main conclusion depends on them.

Is a 0.05 change in correlation compelling? Not by itself, and the paper tests why. In our retraining, the TRIPLEX and MERGE context gains are 0.044–0.078 in mean Pearson rr on smoothed targets (Tables S16, S18). Two checks change what that number means:

  1. Correlation cannot show whether predictions beat a constant, because it ignores shifts and positive rescaling of each gene's predictions. Against the training mean, both TRIPLEX models and all three MERGE variants fall below baseline on raw renal targets; full TRIPLEX scores R2R^2 = −0.054 [−0.106, −0.012] and is below baseline in 19 of 24 patients (Appendix F.9).
  2. The gain depends on what the target contains. On raw spot targets the same gains are 0.016 and 0.011 for TRIPLEX, and 0.004 and 0.006 for MERGE with intervals spanning zero. Keeping the smoothed target fixed and masking the neighbors that enter it removes 77–80% of the TRIPLEX gain (Section 4.1).

Biological magnitude. Agreed: a 0.05 increase in correlation does not by itself establish biological usefulness. Training-mean comparisons and patient-level intervals assess predictive performance and its uncertainty, not biological utility. The paper's contribution is how the interpretation of a gain changes under matched controls: the masking result above shows what the TRIPLEX gain depends on, which needs no threshold for biological importance. Biological utility would require validation against a downstream endpoint, outside this study. The revision states this in Appendix A, and Section 4.6 notes that its ±0.01 equivalence margin was not calibrated to a biologically meaningful effect.

If "0.05" refers instead to the UNI2-H versus ResNet50 contrast gap on measured targets (0.052 for colorectal cell composition and 0.047 for renal genes; Table S7), the colorectal interval includes zero ([−0.028, 0.127]). This is why the revision states the encoder advantage only for renal genes and non-self embedding spaces.

AuthorsOpenReview ↗

Response to Reviewer YeeC (2/3): absolute performance and independence

Is there a meaningful destination, even for a perfect method? For nearby tissue, yes, and the paper measures the distance to it. Three reference points, all on the same queries and patient folds:

  1. The targets are measurable. Split-half reliability is 0.96 for nearby cell composition and 0.77 for nearby gene expression (Table S24), so low scores are not explained by target noise.
  2. A local oracle has a limit that depends on distance. The crop's own measured cell composition predicts the nearby ring (80–160 µm) at R2R^2 = 0.31 and the matched far ring at 0.02 (Table 2). Frozen encoders reach 0.07–0.13 on the nearby ring (Section 4.2), so nearby prediction has clear headroom.
  3. The farther ring (320–640 µm) is predictable once its tissue is in the input. An MLP on tiles inside the ring reaches R2R^2 = 0.26–0.31, which is 0.34–0.40 above the matched 320 µm crop, with every interval above zero (Table S23). With ridge, ring tiles stay below the training mean (Section 4.4).

Predicting the immediate neighborhood from a local image is therefore a reachable goal with measurable headroom. Inferring tissue several hundred micrometres away from a local crop shows little evidence of being reachable. Telling these two apart is what the protocol is for. We agree that the paper evaluates spatial prediction rather than improving it, and the low absolute scores are one of its findings. Without them, a contrast of +0.19 for the farther ring (UNI2-H, MLP) would read as success, although both arms are below the training mean (−0.03 and −0.22; Section 4.2).

Non-independent locations. Locations never enter inference as independent observations; patients are the unit throughout (Sections 3.1, 3.2; Appendix B.3).

  1. Splits are four patient-grouped folds (Xenium 8/4/4 patients, renal 12/6/6), and each Xenium patient's tumor and normal sections stay in the same fold.
  2. Scores reduce each patient's queries to one patient score, and patients are weighted equally regardless of their number of queries.
  3. Uncertainty comes from 20,000 paired patient bootstraps, and p values from exact paired sign-flip tests over the 16 or 24 patients.

The effective sample size is therefore 16 or 24, not the number of locations. Within-patient dependence affects how precisely each patient's score is estimated; the patient-level procedures only require patients to be independent of one another. Split granularity also does not drive the results: moving Xenium from section-level to patient-level splits changes TRIPLEX correlation by −0.007 [−0.018, 0.004] (Table S21). The real cost of 16 and 24 patients is power, which Appendix A states. Some intervals are wide, and on colorectal cell composition the encoder contrasts overlap, which is why the revision states the encoder advantage only where it holds.

AuthorsOpenReview ↗

Response to Reviewer YeeC (3/3): morphology versus expression, and method design

Morphology versus gene expression. Agreed, and the paper keeps the two apart. The colorectal Xenium analyses mainly predict four-class cell composition (epithelial cells, fibroblasts, myeloid cells, lymphocytes), which is close to what H&E shows. The renal Visium analyses and the TRIPLEX and MERGE re-examination predict gene expression. Each target type is reported separately, and the conclusions differ between them:

  1. Encoders. On renal gene expression, every pathology encoder exceeds every natural-image encoder (win rate 79% [70%, 88%]); on colorectal cell composition, the encoders overlap within patient-level uncertainty (Section 4.3).
  2. Spatial supervision. SEAL-UNIv2's gain over its base encoder is neighborhood-specific for cell composition (+0.033, Holm-adjusted p = 0.026) but not for gene expression (Section 4.6).
  3. Expression benchmarks. UNI2-H meets the HEST correlation criterion for 42 of 50 renal genes, yet 14 of the 50 genes and 12 of 24 patients are predicted worse than the training mean (Appendix F.9).

Table 2 tests the link directly. Given the crop's measured 372-gene profile, UNI2-H features still add neighborhood-specific information about cell composition (GG = 0.041 [0.011, 0.074]). When the ring target is its gene profile, the gain is smaller (0.010 [0.006, 0.015]; Appendix F.4). Image features therefore carry information beyond measured expression, more for cell composition than for expression itself, which is consistent with your point. The cell-composition result holds under ridge; with an MLP the estimate is similar (0.048), but its interval crosses zero.

How should this change method design? Three changes, none requiring new data (checklist in Table S4):

  1. Context-aware expression predictors should report the context gain on raw targets, or with the neighbors that enter a smoothed target masked from the input, and score against the training mean.
  2. Encoder selection, including spatially supervised pretraining, should use shared measured targets and per-patient orderings, not prediction of an encoder's own embeddings. The UNI2-H versus ResNet50 difference is 0.303 or −0.002 depending only on the embedding space (Table S8).
  3. Models that enlarge the field of view should be compared with an input that sees the same tissue and with a target-withheld variant, under both a linear and a nonlinear readout, because the answer can differ between readouts (Sections 4.4, 4.5).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

General Response

We thank the AC and the reviewers. All three reviewers found the matched-control analyses useful, and the concerns raised were mainly about controls and presentation. Each is now addressed with new analysis or explicit revision:

  • Does the reduced TRIPLEX/MERGE gain just reflect a harder task? (qHYY, YeeC) New control: with the smoothed target and the task held fixed, masking the neighbors that enter the target removes 77–80% of the TRIPLEX gain. The reduction comes from tissue visibility, not task difficulty.
  • Are spatial locations treated as independent in small cohorts? (YeeC) No. Patients are the unit of inference throughout (patient-grouped folds, paired patient bootstraps, exact sign-flip tests over 16 or 24 patients). Small cohorts cost power, and the revision makes no claim where intervals overlap. On Xenium, moving from section-level to patient-level splits changes TRIPLEX correlation by only −0.007 [−0.018, 0.004].
  • Is predicting morphology the same as predicting gene expression? (YeeC) No, and the paper reports the two separately. Image features add more neighborhood information about cell composition than about gene expression (GG = 0.041 versus 0.010), and the published expression context gains shrink to intervals spanning zero on raw targets.
  • Do the low absolute scores leave no meaningful destination? (YeeC) No. Split-half reliability is 0.96 and 0.77, so the scores are not explained by target noise. A perfect readout of local cell state, the crop's measured cell composition, reaches R2R^2 = 0.31 on the nearby ring versus 0.02 on the matched far ring, while frozen encoders reach 0.07–0.13, so nearby prediction is reachable and has clear headroom.
  • Readability (all reviewers). Section 4 is reorganized around three questions, one per reporting requirement, with a new Table 1 mapping each result to its data and controls.

Reviewer ka76 confirmed the revised scoping of the readout and encoder claims and raised the soundness score. Reviewer qHYY confirmed that most concerns were addressed.

Reviewer YeeC replied that the main concerns were not adequately addressed, without specifying which. The concerns in the review are small cohorts with dependent locations, morphology versus gene expression, and low absolute performance, which the review itself introduces as the usual weaknesses of spatial transcriptomics image analysis. They are limitations of the field, shared by every method in this area, rather than flaws in this paper's design. The paper proposes no new predictor; its contribution is an evaluation protocol, built around these limitations: inference is at the patient level, with the resulting loss of power reported rather than hidden; the two target types are analyzed separately; and low absolute performance is a finding the paper measures against a training-mean baseline and a local oracle. These limitations are also why the protocol matters: when signals are weak and cohorts small, gains reported without a baseline or matched control are easily misread. We would welcome identification of any specific analysis these limitations affect.

On novelty: None of the 27 expression-prediction studies in our sample reports a training-mean baseline, and UNI2-H, which meets the HEST criterion for 42 of 50 genes, predicts 12 of 24 patients worse than that baseline. The three reporting requirements need no new data or models and apply directly to existing benchmarks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.