acceptodds
Under review as a conference paper at ICLR 2027

Reassessing Spatial Context in Histopathology

Abstract

Histopathology classification often depends on both local morphology and surrounding tissue architecture, yet effectively incorporating spatial context into patch-level prediction remains challenging. We study contextual patch classification using a shared transformer that jointly processes and supervises a center patch and its spatial neighborhood, encouraging representations that preserve local evidence while exploiting broader tissue context. We further investigate a cross-slide contextual-update regularizer designed to constrain how neighborhood information modifies center predictions. Across 64,921 test patches, two frozen pathology foundation-model encoders, and three seeds, our dual-view training recipe improves slide-bootstrap ensemble macro-F1 over a strong contextual baseline by 2.03 percentage points with UNI2-h (95% CI , ) and 0.86 points with ALICE (95% CI , ). The additional cross-slide regularizer provides no further improvement. These results demonstrate that jointly learning from local and spatially contextualized views provides a simple and effective approach to contextual histopathology classification, with consistent gains across foundation-model representations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Histopathology

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_XEGFOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper reassesses the role of spatial context in histopathology patch classification. The authors identify substantial WSI-level overlap between the stored training and test partitions of the SPIDER dataset and construct a new WSI-disjoint evaluation split. They then compare a full-view contextual transformer, a shared dual-view model supervised on both center and full spatial views, and a variant with a cross-slide conditional-moment regularizer. Experiments use two frozen pathology encoders, three training seeds, and WSI-level bootstrap evaluation. The dual-view training recipe improves macro-F1 over the full-view baseline, particularly with UNI2-h. However, the comparison changes several training components simultaneously, and the proposed moment regularization does not improve the primary metric.

Critical strengths

The authors identify substantial WSI overlap between the training and test partitions of the SPIDER dataset, raising an important concern about the reliability of the original evaluation.

The paper establishes a WSI-disjoint split and evaluates the models using multiple encoders, training seeds, and WSI-level bootstrapping, providing a more reliable evaluation protocol.

Critical weaknesses

The full-view and dual-view models differ not only in supervision but also in normalization and loss aggregation, making it difficult to attribute the reported improvements to the proposed learning method.

The proposed moment regularization fails to improve the primary performance metric for either encoder, and its effect on conditional coherence is not directly evaluated.

The theoretical analysis largely relies on the established tower property, while the practical benefit of the proposed cross-slide estimator for learning conditional coherence remains unclear.

All experiments are conducted on the SPIDER dataset, limiting the evidence that the proposed method generalizes to other datasets or data distributions.

Questions for authors

See a Weaknesses

1 comment

e thank Reviewer XEGF for the concise review and for recognizing the audit and the protocol.

1. Confounded comparison. Agreed. One factor, loss aggregation, is removed: on our balanced batches the hierarchical loss equals the sample mean. View supervision and normalization order still change together (Table 1), and the paper states that it measures a training recipe. Sec. 5.2 now reports the closest matched controls we have, the endpoints of the existing single-seed sweep, with their limits. Full-budget matched runs are the right next step, and we did not have enough time to run them.

2. The moment penalty does not improve F1, and its effect on coherence is not evaluated. Agreed. We did not log held-out moment values, the penalty's gradient relative to cross-entropy, or a range of λ, and we did not have enough time to rerun training with them. Sec. 5.3 now says what this means: the null cannot separate "coherence improved but classification did not" from "the penalty had little practical influence". Appendix B.4 adds one reason a decrease of the penalty need not mean a lower pooled moment (between-slide cancellation).

3. Theory relies on the tower property; practical benefit of the estimator is unclear. Agreed on the first point. The paper presents the tower property as standard. The contribution is the application: coherence is not pointwise agreement, so a pointwise penalty can oppose the Brier gain from context (Proposition 1). The estimator's benefit is not shown empirically. Appendix B.3 verifies its sampling target on synthetic batches. We report the negative result, and B.4 shows its target differs from the pooled moment.

4. SPIDER only. Agreed, and the Limitations state it. We wanted to focus on the audit and the evaluation question first, and we did not have enough time to evaluate other datasets.

Thank you again.

Reviewer_EbDWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper reassesses spatial context modeling for histopathology patch classification on SPIDER. It identifies substantial WSI overlap in the original train/test split and constructs a WSI-disjoint evaluation protocol. The authors further study dual-view contextual training and propose a cross-slide conditional-moment regularizer based on conditional coherence. Experiments with frozen ALICE and UNI2-h features show that the dual-view training recipe can improve performance.

Critical strengths

None

Critical weaknesses
  1. The presentation quality is currently a major weakness. The manuscript contains many incomplete descriptions, grammatical issues, and awkward expressions, and several sections lack a clear logical flow. These problems make it unnecessarily difficult to understand the exact methodological contributions and to assess whether the experimental evidence supports the authors’ claims. I believe the paper needs substantial reorganization and thorough language revision, rather than only local proofreading.

  2. The main concern is that the technical novelty appears limited. The probabilistic formulation largely builds on existing conditional-coherence and kernel-moment principles, while the proposed cross-slide regularizer does not improve the primary metric. The strongest contribution seems to be the audit of WSI overlap in the SPIDER split rather than a new learning method. Moreover, the main Full-view vs. Dual-view comparison changes multiple factors simultaneously, including view supervision, normalization order, and loss aggregation, making it difficult to attribute the observed gains to the proposed mechanism.

  3. The main comparison is confounded. Full-view and Dual-view differ in supervision, normalization, and loss aggregation, so the gain cannot be attributed specifically to the dual-view mechanism.

  4. The experimental scope is narrow. The study uses only SPIDER, and the two frozen encoders are evaluated on the same cohort.

  5. WSI-disjoint splitting does not prevent higher-level leakage. Patient, institution, scanner, and stain-batch identifiers are unavailable, so broader independence cannot be guaranteed.

  6. The performance drop on the reconstructed split cannot be interpreted as overlap-induced inflation. The original and new splits differ in overall data composition, not only WSI overlap.

  7. The statistical and experimental budget is limited. The main results use only three seeds, and the bootstrap analysis does not account for training randomness.

Questions for authors

Please respond to the Critical Weaknesses one by one.

2 comments

We thank Reviewer EbDW for the detailed list of concerns. We understand that the presentation made the paper hard to follow, and we respond to each point in order.

1. Presentation. In the revision, a "Setting and terms" paragraph defines target, center view, full view, frozen encoder, and WSI-disjoint split before first use. A new Table 1 lists, for each system, the supervised views, normalization order, and extra loss term. The contribution list is reordered, the appendices are in the main PDF, and passages that described loss aggregation as a difference are corrected. We did not have time for a full reorganization. We would be grateful for the specific passages you found incomplete, so we can fix them.

2. Limited novelty. We agree that the formulation builds on existing principles, and the paper says so (Introduction and Related Work). The regularizer does not improve the primary metric, and we report that. We agree the strongest contribution is the audit, and the contribution list now says so. The coherence-versus-invariance distinction is presented as an application, with its negative result.

3. Confounded main comparison. Agreed. One factor is removed: on our balanced batches (64 WSIs per organ, 2 targets per WSI), the WSI- and organ-averaged loss is exactly the sample mean (Eq. 6 gives each target weight 1/512). The comparison still changes view supervision and normalization order together. Table 1 lists them, and the paper states that it measures a training recipe. The closest matched controls we have are the endpoints of the existing single-seed sweep, reported in Sec. 5.2 with their limits. Full-budget, multi-seed matched runs are what the question needs. We chose to put our effort into the audit, and we did not have enough time to run them.

4. Narrow scope (SPIDER only). Agreed, and the Limitations state it. We did not have enough time to evaluate other datasets. The audit itself needs only slide identifiers and stored split labels, so the new script can be applied to any dataset that provides them.

5. Higher-level leakage. Agreed. Patient, institution, scanner, and stain keys are not in the records, so WSI is the strictest grouping we can verify (Sec. 4.1 and the Limitations). Sec. 5.1 now states that the audit is one-sided. Overlap shows that a split is not slide-disjoint. Its absence in our split does not show independence at those levels.

6. The performance drop is not overlap-induced inflation. Agreed, and the paper already says so (Sec. 5.1: the 4.64 and 7.08 point differences "do not estimate the amount of inflation caused by overlap alone"). We make no inflation estimate. The claim is that the stored test set cannot measure generalization to unseen WSIs.

7. Seeds and bootstrap. Agreed. The intervals describe test-WSI sampling and not training randomness (Sec. 4.3 and the Limitations). New descriptive information: every Dual-view seed exceeds every Full-view seed for both encoders (lowest Dual-view versus highest Full-view F1: 0.8912 versus 0.8869 for ALICE, 0.8817 versus 0.8579 for UNI2-h). With three seeds per arm the smallest attainable exact two-sided permutation p-value is 0.10, so we do not offer this as a test. We did not have enough time to add seeds.

We hope the revision makes the claims clearer. Thank you again.

I thank the authors for their response. Given the current quality of the manuscript and the research question it addresses, I stand by my original score.

Reviewer_er8hOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

There are two distinct contributions of this manuscript. The first is an audit of the SPIDER records used by the authors: 2,436 WSI identifiers appear in both stored training and test partitions, accounting for 84.59% of stored test targets. The authors construct a WSI-disjoint replacement split covering 408,235 targets from 6,785 slides. The second is an investigation of contextual prediction using frozen ALICE and UNI2-h features: a full-neighborhood classifier is compared with a shared center/full-view classifier and a version with cross-slide conditional-moment regularization.

The dual-view training recipe improves three-seed ensemble macro-F1 by 2.03 percentage points for UNI2-h, while the smaller ALICE improvement has a confidence interval spanning zero. The proposed moment penalty does not improve the primary metric. Importantly, the authors acknowledge that normalization and supervision change together, so the positive comparison does not isolate center-view supervision. I therefore read this primarily as a benchmark-audit and empirical reassessment paper, rather than a demonstration of a superior contextual-learning objective.

Critical strengths
  1. The audit addresses a problem that would materially change how the benchmark should be used. The distinction between classifying new patches from familiar slides and generalizing to unseen slides is central to this evaluation. The authors appropriately restrict their finding to the records examined. They also avoid attributing the entire accuracy reduction after re-splitting to leakage, since the training and test compositions change. That qualification strengthens, rather than weakens, the audit.

  2. The evaluation does not mistake a large patch count for a large number of independent observations. Paired resampling occurs at the WSI level within organ, and the manuscript explicitly states that its intervals describe test-slide sampling—not uncertainty over model training. Reporting three-seed results separately from probability-ensemble contrasts makes the evidence easier to interpret.

  3. The distinction between coherence and invariance is useful. Proposition 1 explains why additional context can legitimately change a prediction: the refined Bayes posterior must average back to the coarse posterior, not equal it on every example. The authors correctly present this as an application of established principles. The paper also retains the negative regularization result instead of treating small descriptive calibration changes as evidence of success.

Critical weaknesses
  1. “Release-specific” is an appropriate qualification, but the release must then be identifiable. The supplied manuscript does not provide an immutable dataset revision, the precise identifier-extraction procedure, or representative overlapping records that would let another group reproduce the headline count. It refers to Appendices A, F, and H, but the supplied PDF ends with the references. The reproducibility statement says that audit records and model outputs were retained; I could not assess those materials.

  2. A separate check of SPIDER’s public dataset documentation confirms that it explicitly promises slide-disjoint partitions. This makes the reported discrepancy important, but also makes it necessary to distinguish an upstream release problem from identifier parsing or local processing. The public documentation alone cannot resolve that distinction. A small executable audit with pinned inputs would be more valuable here than additional model comparisons.

  3. The spatial-context question remains entangled with implementation choices. The principal comparison evaluates two systems that both use the full neighborhood at inference. It therefore measures a training-recipe difference, not the predictive value of context over a comparably trained center-only model. Pre-LN versus post-LN normalization also changes. The authors acknowledge this, but the shorter, single-seed sweep does not settle the issue: at �2��2�, performance is below �3��3� for both encoders in that experiment. A matched, full-budget comparison is needed to determine whether dual-view supervision contributes to the reported gain.

  4. There is also a concrete inconsistency in the description of loss aggregation. Under the stated batch construction - four organs, 64 WSIs per organ, and two targets per WSI—Equation 6 assigns each target weight $

14×64×2=1512.\frac{1}{4\times64\times2}=\frac{1}{512}.

$ At �4��4�, this is exactly ordinary sample-averaged full-view cross-entropy on the same batch. Unless the implementation uses unequal counts or additional weights, hierarchical averaging is not a separate difference between the systems. This should be reconciled before interpreting the comparison.

  1. The regularization null is reported clearly, but not explained experimentally. Proposition 2 establishes unbiased estimation of a distinct-WSI finite-population cross-product. As the authors correctly note, this is not automatically the squared population moment in Equation 7. Writing �5��5� for a slide’s mean feature-weighted update, the complete distinct-slide quantity satisfies $
J_{\neq}$= $\|\bar\mu\|^2 ] \neg \frac{1}{G(G-1)} \sum_{g=1}^{G}\|\mu_g-\bar\mu\|^2

$

Thus, for a fixed collection of slides, reducing this objective need not mean reducing the pooled squared moment; between-slide cancellation also matters. This follows from the estimator’s algebra and is not an allegation of an implementation error.

  1. Negative U-statistic estimates are not inherently problematic. What is missing is evidence about the regularizer’s behavior during learning: does it reduce a held-out coherence discrepancy, contribute appreciable gradients, or behave differently over a reasonable weight range? Without those diagnostics, unchanged F1 cannot distinguish “coherence improved but classification did not” from “the penalty had little practical influence.” The reported null remains valid for the tested configurations, but its methodological lesson is limited.
Questions for authors

1.Can the exact SPIDER revisions be provided, a minimal overlap-checking script, and example records establishing that shared identifiers refer to the same source WSIs? Are the referenced appendices and split manifests available separately?

  1. Does the implemented hierarchical loss reduce to the ordinary sample mean under the reported balanced batches? Are full-budget, three-seed results available for a matched post-LN full-view model and a genuinely center-only model?

  2. Does the moment penalty measurably reduce held-out moment discrepancy, and how large is its gradient contribution relative to cross-entropy? This would substantially affect how I interpret the negative result.

1 comment

We thank Reviewer er8h for a careful and constructive review, and for reading the paper as primarily a benchmark audit. We address each point.

Appendices missing from the PDF. We apologize. The appendices were in the separate supplementary PDF, and the main PDF ended with the references. The revised main PDF includes them.

Q1: reproducing the headline count. We added Appendix A.1 and an executable check:

  • Revisions. Table 5 lists the head revision of main for each of the four SPIDER repositories (all last modified between March and April 2025, before our downloads), for example histai/SPIDER-skin at 7ab1badb…4b51.
  • Identifier. An identifier is the organ prefix plus the stored slide_id. A WSI overlaps if it has both stored-train and stored-test targets.
  • Script. audit_wsi_overlap.py is about 50 lines (standard library plus NumPy, with a self-test) and is listed verbatim in the appendix. It uses only the stored split labels, with no splitting, filtering, or identifier normalization.
  • Reproduction on skin. Recomputed from the stored skin records: 3,780 WSIs, 1,348 in both partitions, 22,216 test targets in them, 77.43%. All equal Table 3. One difference: the records hold 28,690 stored-test targets, two fewer than the 28,692 in the table. We have not traced those two rows, and the appendix says so. Example WSIs, with stored train/test target counts: slide_0001 (51/45), slide_0002 (173/12), slide_0004 (6/2).
  • Same physical slide. In the skin records, 2,974 of 28,690 stored-test windows (10.37%) contain a patch that also lies in a stored-train window. All belong to overlapping WSIs; none of the 6,474 test windows from other WSIs does. None of the ~2.7 million patch rows occurs under two slide identifiers. Most overlapping test windows share no patch, so this bounds literal pixel reuse and does not measure leakage through slide-level appearance.

Only skin could be rechecked in the discussion period. The other three organs rely on the stored records analyzed for the submission, as the Limitations now say. We did not have time to prepare an anonymized release of the split manifest; per-organ counts are in Appendix Table 4.

Release versus parsing or processing. We agree the public documentation cannot settle this. The script takes the stored labels as given, which removes our own splitting and filtering from the picture, but it cannot tell whether the overlap originates in the release or in how identifiers are encoded in it. The paper therefore says "release-specific" and states this limit. We intend to share the audit with the dataset maintainers. Q2: loss aggregation and matched controls. You are right about the loss. With 64 WSIs per organ, 4 organs, and 2 targets per WSI, Eq. 6 gives each target weight 1/512, which is the ordinary sample mean. The hierarchical form matters only for unbalanced batches. We corrected the abstract, Introduction, Sec. 3.2, 4.2, 5.2, Discussion, and Appendix G: the two systems differ in view supervision and normalization order. That reduces the confound from three factors to two, but does not remove it.

Full-budget, three-seed matched controls are not available, and we did not have enough time to run them. The existing sweep does contain the closest controls, and we now report them (Sec. 5.2). At α=0 the post-LN model is supervised only on the full view; the center-view column at α=1 is a center-only model of the same architecture and budget. For ALICE, full-view-only (0.8781) exceeds center-only (0.8581). For UNI2-h, center-only (0.8564) is higher than full-view-only (0.8486). These are single-seed, test-evaluated values under the shorter budget, so they are not comparable with Table 2, and the UNI2-h gap (0.8 points) is of the order of the seed standard deviations there (0.4 to 0.9). They show neither that the full neighborhood beats a center-only model for UNI2-h, nor that it does not.

Q3: the regularizer. Thank you. Your algebra is correct, and Appendix B.4 now states it. With per-WSI mean updates a_g and mean ā, the distinct-WSI target equals G/(G−1)‖ā‖² − (1/(G−1))·mean‖a_g‖². It can fall through larger per-WSI updates that cancel across WSIs, with no fall in the pooled squared moment. We did not log the held-out moment value, the penalty's gradient norm relative to cross-entropy, or performance across a range of λ, and we did not have enough time to rerun training with them. Sec. 5.3 and the Limitations now say the null cannot separate "coherence improved but classification did not" from "the penalty had little practical influence".

We wanted to put the audit and the coherence-versus-invariance distinction on a clean footing. Your suggestion of a small executable audit with pinned inputs was the one we valued most, and we hope Appendix A.1 goes some way toward it.

Comments on the paper

1 comment

General response to all reviewers

We thank the reviewers for their time and careful reading. We were glad that Reviewer er8h found that the audit "would materially change how the benchmark should be used" and valued that we restrict it to the records examined, resample at the WSI level, and keep the negative regularization result. We were glad that Reviewer XEGF found that it raises an important concern about the original evaluation. Reviewer EbDW listed no strengths; we thank them for the detailed list of concerns and took the presentation concern seriously. We have uploaded a revised PDF with changes in blue. Individual responses follow each review. Four themes recur.

1. What the paper claims. All three reviews read the audit as the main contribution, and we agree. The contribution list now starts with it. The Full-view versus Dual-view comparison is presented as what it is: a comparison of two training recipes, not a test of center-view supervision alone and not a test of context versus no context.

2. Our focus, and the experiments we did not run. We wanted to focus on the idea itself: that evaluation on a stored split can silently measure same-slide generalization, and that "context helps" should be stated as conditional coherence and not as agreement between predictions. We put our effort into stating this precisely and into reporting the negative regularization result, not into extensive validation. We recognize that this leaves real gaps. The reviewers named them precisely and constructively, and we agree they are the right next experiments. We did not have enough time to run them during the discussion period:

  • full-budget, three-seed controls: a post-LN full-view-only model and a center-only model;
  • diagnostics of the moment penalty during training (held-out moment value, gradient contribution relative to cross-entropy, a range of λ);
  • datasets other than SPIDER;
  • more seeds, and uncertainty that includes training randomness.

The claims are scoped to what was run, and the Limitations now say so. Patient, institution, scanner, and stain keys are not in the records, so we cannot add grouping at those levels.

3. What we could do without new training.

  • Loss aggregation. Reviewer er8h is right. Under our balanced batches, the hierarchical loss equals the sample mean (details in the reply to er8h). We corrected the manuscript. The two systems differ in view supervision and normalization order.
  • Matched controls. The existing single-seed view-weight sweep contains the closest available post-LN full-view-only and center-only models. We now report their numbers and their limits (Sec. 5.2).
  • Estimator. Appendix B.4 gives an identity showing that the distinct-WSI target can decrease without the pooled squared moment decreasing.
  • Audit reproducibility. Appendix A.1 gives the repository revisions, an executable overlap script, and a reproduction of the skin counts from stored records.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.