acceptodds
Under review as a conference paper at ICLR 2027

RePath: Zero-Shot Whole-Slide Tumor Segmentation via Intra-Slide Relational Affinity

Abstract

Zero-shot tumor segmentation of whole-slide images (WSIs) via vision–language models (VLMs) faces a fundamental obstacle: the gigapixel resolution of WSIs forces patch-level processing, with each patch independently scored against fixed text embeddings. This absolute patch-level scoring produces two failure modes: prompt dependency (sensitivity to prompt wording) and spatial inconsistency (adjacent patches of the same tissue receiving conflicting labels), both reflecting a mismatch between slide-level histopathological interpretation and independent patch-wise classification. We propose RePath, which reformulates inference from per-patch scoring against fixed text embeddings to relational comparison among patches within the slide, built on three design principles. First, slide-specific anchor selection draws tumor, normal, and non-tissue anchor patches from the slide itself, using text prompts only to seed the selection. Second, intra-slide relational affinity recasts isolated image-text similarity as the Jensen-Shannon Divergence (JSD) between each patch's class distribution and the anchor distributions, yielding tumor, normal, and non-tissue affinity maps. Third, Differential Affinity Integration (DAI), inspired by clinical differential diagnosis, contrasts the tumor affinity against the normal and non-tissue affinities to produce a coherent segmentation mask. On 1,647 test WSIs across four organs and three VLM backbones, RePath improves Dice by 13.0 and IoU by 14.5 over the vanilla zero-shot baseline and, on average, matches or exceeds few-shot supervised methods trained with 10 labeled WSIs, without annotation or training.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Zero-Shot Tumor Segmentation, whole-slide images, vision–language model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QJM3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RePath is a training-free method for whole-slide tumor segmentation using frozen histopathology VLMs. It selects tumor, normal, and non-tissue anchors from each slide, compares patches through Jensen–Shannon divergence over text-conditioned class distributions, and assigns tumor labels by contrasting category-specific affinities. Evaluation across four primary datasets and three backbones shows consistent improvements over vanilla zero-shot inference and competitive performance against methods using ten annotated slides. The appendix adds cross-center evaluation, smoothing baselines, and additional content.

Critical strengths

1.The method addresses a practical limitation of patch-wise VLM inference with a relatively simple procedure. Improvements over vanilla inference are consistent across all three backbones; for CONCH, average IoU increases from 49.9 to 63.6 across the four primary datasets. 2.The evaluation goes beyond comparisons with basic zero-shot methods. The appendix includes graph smoothing and label propagation, cosine-affinity ablations, additional segmentation datasets, and comparisons with weakly and fully supervised models. These experiments make the empirical contribution more convincing. 3.The prompt robustness analysis is useful. Repeating evaluation across ten prompt sets shows both higher average performance and lower variability, while the anchor-budget and sampling experiments support the use of a small anchor set.

Critical weaknesses

1.The clinical applicability is narrower than parts of the paper suggest. Appendix I reports only 8.6% specificity on tumor-negative Camelyon16 slides, and the main evaluation is restricted to tumor-bearing slides. The proposed screening-then-segmentation workflow is not evaluated. This does not invalidate the segmentation results, but statements about immediate deployment and robustness in clinical environments should be qualified. 2.The methodological advance needs more precise positioning. The paper explicitly builds on CDAM’s class-distribution observation, while the final DAI decision is equivalent to choosing tumor when its normalized affinity exceeds both competing affinities. The contribution therefore appears to lie primarily in the complete slide-specific inference procedure. In addition, the text does not clearly distinguish category-wise normalization from a shared normalization across all three maps; this distinction changes the classifier. Appendix G.8 also refers to the “same adjacency matrix” as RePath, although no adjacency graph appears in the proposed formulation. 3.Some evaluation details limit confidence in the finer claims. The paper states that hyperparameters are fixed across datasets but does not explain what data were used to select them initially. The significance tests quantify variation across prompts on a fixed test set, rather than uncertainty across patients or slides. There are also concrete reporting inconsistencies: Table A11 lists 35 small-tumor slides, whereas the accompanying text uses 34, and its detection percentages do not match the tabulated counts. The 0.95-second anchor-selection time in Table A9 also needs reconciliation with the 0.24-second inference time in Table A6.

Questions for authors

1.Are the affinity maps standardized separately for each category or jointly? Please provide the exact normalization formula and explain how empty normal or non-tissue anchor sets are handled. What adjacency matrix is being referenced in Appendix G.8? 2.What data informed the choice of the confidence threshold, pooling temperature, and backbone-specific softmax temperatures? Were these choices finalized before evaluating the reported test sets? 3.Can you reconcile the counts and detection rates in Table A11 and the timings in Tables A6 and A9? Please also provide paired slide-level confidence intervals for the main improvements, accounting for patient grouping where applicable

7 comments

We sincerely thank the reviewer for taking the time to carefully review our paper. We especially appreciate the reviewer's recognition that RePath addresses a practical limitation of patch-wise VLM inference through a relatively simple procedure, achieving consistent improvements across all three VLM backbones. We also appreciate the acknowledgment of our comprehensive evaluations, including additional baselines, ablation studies, and robustness analyses. We carefully address each concern below.

[W1] Clarification of Clinical Applicability and Evaluation Scope. The clinical applicability is narrower than parts of the paper suggest. Appendix I reports only 8.6% specificity on tumor-negative Camelyon16 slides, and the main evaluation is restricted to tumor-bearing slides. The proposed screening-then-segmentation workflow is not evaluated. This does not invalidate the segmentation results, but statements about immediate deployment and robustness in clinical environments should be qualified.

RePath is designed for tumor localization in tumor-bearing WSIs, rather than standalone tumor screening. As reported in Appendix I, its performance on tumor-negative slides indicates that RePath alone is insufficient for reliable slide-level screening. Accordingly, our main segmentation results should be interpreted as performance on tumor-bearing WSIs.

A potential approach is to separate slide-level tumor classification from tumor localization. Established methods, including MIL-based classifiers [R1–R2] and zero-shot approaches such as MI-Zero [R3], could provide initial screening, followed by RePath for tumor segmentation. This complete two-stage workflow has not been evaluated in our study.

We have clarified the intended scope in the Introduction (p. 1, lines 44–46) and the potential two-stage workflow and its limitations in Appendix I (p. 29, lines 1548–1551) and qualified statements regarding clinical feasibility and deployment in Appendix G.4 (p. 21, line 1121).

[R1] TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification, NeurIPS, 2021.

[R2] Feature Re-embedding: Towards Foundation Model-Level Performance in Computational Pathology, CVPR, 2024.

[R3] Visual language pretrained multiple instance zero-shot transfer for histopathology images, CVPR, 2023.

[W2 / Q1] Methodological Contribution and Implementation Details.

W2. The methodological advance needs more precise positioning. The paper explicitly builds on CDAM’s class-distribution observation, while the final DAI decision is equivalent to choosing tumor when its normalized affinity exceeds both competing affinities. The contribution therefore appears to lie primarily in the complete slide-specific inference procedure. In addition, the text does not clearly distinguish category-wise normalization from a shared normalization across all three maps; this distinction changes the classifier. Appendix G.8 also refers to the “same adjacency matrix” as RePath, although no adjacency graph appears in the proposed formulation.

Q1. Are the affinity maps standardized separately for each category or jointly? Please provide the exact normalization formula and explain how empty normal or non-tissue anchor sets are handled. What adjacency matrix is being referenced in Appendix G.8?

Methodological contribution. RePath's primary contribution is its complete slide-specific inference procedure, which reformulates independent patch–text classification as intra-slide relational comparison. Specifically, RePath extends CDAM's class-distribution observation to WSI-level inference through automatically selected slide-specific anchors and integrates the resulting tumor, normal, and non-tissue relational affinities using DAI. While DAI's final binary decision is mathematically equivalent to a three-way affinity comparison, it operates on slide-specific relational affinities rather than direct patch–text similarities.

We have clarified this contribution in the Introduction (p. 2, lines 72–73) and the role and mathematical equivalence of DAI in Sec. 4.3 (pp. 6-7, lines 321–338).

Shared normalization and empty anchor handling. The three affinity maps are normalized jointly, not separately. For each WSI containing NN patches, a shared mean μ\mu and standard deviation σ\sigma are computed over all 3N3N raw affinity scores:

\qquad \sigma = \sqrt{\frac{1}{3N-1}\sum_{m\in\{T,N,J\}}\sum_{i=1}^{N}(R_m(x_i)-\mu)^2}.$$ The normalized affinities are $$\tilde{R}_m(x_i)=\operatorname{sigmoid}\left( \frac{R_m(x_i)-\mu}{\max(\sigma,0.05)+10^{-8}} \right), \quad m\in\{T,N,J\}.$$ Here, the standard deviation is lower-bounded by $0.05$ to prevent excessive amplification of small score variations, and $10^{-8}$ ensures numerical stability. If the normal or non-tissue anchor set is empty, its raw affinity map is set to zero before shared normalization. These zero-valued maps remain included when computing $\mu$ and $\sigma$. We have clarified the shared normalization in Sec. 4.2 (p. 6, lines 299–301) and provided the complete formula and empty-anchor handling in Appendix G.12 (p. 25, lines 1299–1322). **Clarification of the adjacency matrix.** RePath does not construct or use an adjacency matrix. The adjacency matrix mentioned in Appendix G.8 refers to the patch-neighborhood graph used by the graph-based smoothing and label propagation baselines, rather than RePath. We have revised Appendix G.8 (p. 23, lines 1201–1202) to clarify this ambiguity.

[W3 / Q2 / Q3] Evaluation Details and Statistical Reliability.

W3. Some evaluation details limit confidence in the finer claims. The paper states that hyperparameters are fixed across datasets but does not explain what data were used to select them initially. The significance tests quantify variation across prompts on a fixed test set, rather than uncertainty across patients or slides. There are also concrete reporting inconsistencies: Table A11 lists 35 small-tumor slides, whereas the accompanying text uses 34, and its detection percentages do not match the tabulated counts. The 0.95-second anchor-selection time in Table A9 also needs reconciliation with the 0.24-second inference time in Table A6.

Q2. What data informed the choice of the confidence threshold, pooling temperature, and backbone-specific softmax temperatures? Were these choices finalized before evaluating the reported test sets?

Q3. Can you reconcile the counts and detection rates in Table A11 and the timings in Tables A6 and A9? Please also provide paired slide-level confidence intervals for the main improvements, accounting for patient grouping where applicable.

Correction of the Size-Stratified Analysis (Table A11 → A12 in the revised manuscript). One of the 35 small-tumor slides in the Camelyon16 test set was incorrectly processed in the previous analysis. We corrected this error and repeated the evaluation using a newly generated prompt set. The updated results include all 48 tumor-bearing slides, including 35 small-tumor slides. Both RePath and Vanilla detect tumors in 34/35 small-tumor slides (97.1%), while RePath improves IoU from 17.4% to 32.7%. We have updated the table and its accompanying discussion accordingly.

Correction of Anchor Selection Time (Table A9 → A10 in the revised manuscript). We identified that the previously reported 0.95 s/WSI for uniform anchor selection included additional operations beyond anchor selection, such as evaluation-related computations. After re-examining the timing records and conducting stage-specific measurements, the corrected uniform anchor selection time is approximately 0.0018 s/WSI, while the k-means selection time remains 8.24 s/WSI. This clarifies the discrepancy with the 0.24 s/WSI inference time reported in Table A6. We have corrected the table accordingly.

Hyperparameter selection. We aimed to use a common hyperparameter configuration across organs and cohorts, rather than optimizing parameters for individual datasets. The confidence threshold (δ=0.2\delta=0.2) was selected based on cross-dataset sensitivity analysis on the reported evaluation datasets (Table A3). The pooling temperature (β=0.5\beta=0.5) was initially chosen as the midpoint of [0,1][0,1], with additional analysis showing at most 1.1 percentage points of variation in average IoU across four datasets (β∈[0.1,1.0]\beta\in[0.1,1.0]; Appendix G.20). Backbone-specific softmax temperatures (τ\tau) were chosen empirically based on class-distribution sharpness. These settings were selected using the reported evaluation datasets and applied consistently without dataset-specific tuning.

Paired slide-level confidence intervals. We additionally computed paired 95% bootstrap confidence intervals using 5,000 slide-level resamples. Results were obtained from an independent run with a different LLM-generated prompt set and therefore differ from those in Table 1. Patient identifiers were unavailable in the evaluation datasets; therefore, the analysis was conducted at the slide level.

Table R1. Paired slide-level bootstrap confidence intervals for RePath–Vanilla IoU improvements (percentage points). Results are from an independent run with a different LLM-generated prompt set and thus differ slightly from Table 1.

DatasetMean ΔIoU95% CI
CM16+24.92[17.22, 32.78]
PAIP Prostate+12.21[11.06, 13.42]
PAIP HCC+16.96[15.23, 18.69]
PAIP Colon+8.00[7.09, 8.93]

All four confidence intervals exclude zero, supporting positive mean IoU improvements across the evaluated slides.

We have corrected the size-stratified analysis (Table A11 → A12 in the revised manuscript; p. 24, lines 1277–1297) and anchor selection timing (Table A9 → A10; pp. 23–24, lines 1234–1254), updated the corresponding results in the Discussion (p. 10, line 490), clarified the hyperparameter selection procedure (p. 7, lines 356–361; Appendix G.1), and added the pooling-temperature sensitivity analysis (Appendix G.20, p. 28, lines 1482–1496) and paired slide-level confidence intervals (Appendix G.21, pp. 28–29, lines 1499–1519).

Reviewer_51P7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RePath for zero-shot whole-slide tumor segmentation. The core idea is to re-score each patch through relational affinity over class distributions, using tumor, normal, and non-tissue anchor patches drawn from the same WSI. This alleviates the prompt dependency and spatial inconsistency of conventional patch-wise VLM inference. The method consists of slide-specific anchor selection, JSD-based affinity computation, and DAI. The paper reports fairly comprehensive experiments, with detailed comparisons on four pathology datasets and three VLM backbones.

Critical strengths
  1. The motivation is novel and reasonable, and the idea is distinctive. Turning independent image-text classification into relational comparison within the same WSI is reasonable from the perspective of clinical diagnosis.
  2. The improvement is significant. The average IoU is clearly higher than that of the zero-shot baselines, and higher than that of the 10-WSI few-shot baselines on all three backbones.
Critical weaknesses
  1. The JSD ablation does not strictly control variables, because it does not separate the contributions of the representation space and the distance function. In the appendix (Table A10), the authors compare

    • RePath anchors + JSD + DAI
    • RePath anchors + feature-level cosine similarity + DAI

    These two configurations differ in two factors at once.

    • Representation space. The visual feature space versus the text-conditioned class-distribution space.
    • Distance or similarity metric. Cosine similarity versus JSD.

    I suggest that the authors add a more strictly controlled ablation that keeps the same anchors, the same class-distribution representation, the same pooling, and the same DAI. This would verify more accurately that JSD is the core component of the design.

  2. DAI is described as a new differential-diagnosis-inspired integration mechanism. However, at the final binary segmentation step, the current formula is mathematically equivalent to checking whether the normalized tumor affinity is the largest of the three classes. The DAI score is positive exactly when the normalized tumor affinity is larger than both the normalized normal affinity and the normalized non-tissue affinity. I suggest that the authors explain what DAI contributes beyond this three-way comparison, or describe it accordingly.

  3. The definition of the z-score normalization is not clear enough. This detail may directly determine whether DAI actually changes the decision boundary relative to a plain argmax over the raw affinities. I suggest that the authors provide pseudocode or write out the full computation.

  4. RePath relies on high-confidence tumor anchors selected from the test slide itself, which depends heavily on the accuracy of the VLM predictions. What happens if the tumor anchors are wrong? For example, under domain shift the VLM may produce false-positive predictions. How would the authors handle this case? The paper includes related experiments, namely tumor anchors drawn from ground-truth regions (Table 4), fully random anchors (Table A10), generic or incorrect keywords (Table A12), and tumor-negative slides (Appendix I). These cover fully correct, fully random, or fully wrong anchors, and wrong keywords. None of them increases the fraction of wrong anchors step by step. I suggest a controlled experiment that gradually introduces mislabeled anchors, to test whether the final results remain robust.

Questions for authors

Please refer to the weaknesses

5 comments

We sincerely thank the reviewer for the careful assessment of our work and the constructive feedback. We are particularly encouraged that the reviewer found RePath's motivation novel and its slide-specific relational inference distinctive, especially in connection with clinical diagnostic reasoning, and recognized its substantial improvements over both zero-shot and 10-WSI few-shot baselines across all three VLM backbones. We address the reviewer's valuable comments below.

[W1] Controlled Ablation of JSD vs. Cosine Similarity. The JSD ablation does not strictly control variables, because it does not separate the contributions of the representation space and the distance function. In the appendix (Table A10), the authors compare • RePath anchors + JSD + DAI • RePath anchors + feature-level cosine similarity + DAI These two configurations differ in two factors at once. • Representation space. The visual feature space versus the text-conditioned class-distribution space. • Distance or similarity metric. Cosine similarity versus JSD. I suggest that the authors add a more strictly controlled ablation that keeps the same anchors, the same class-distribution representation, the same pooling, and the same DAI. This would verify more accurately that JSD is the core component of the design.

We conducted an additional controlled ablation comparing cosine similarity and JSD in the same class-distribution space, while keeping anchor selection, affinity pooling, and DAI unchanged. Feature-level cosine similarity is also included for reference.

Table R1. Controlled comparison of similarity metrics (IoU, %). All methods use identical anchor selection, affinity pooling, and DAI. The class-distribution cosine similarity and JSD variants differ only in the similarity metric. Results are from a single independent run, and thus differ slightly from Table 1.

Similarity MetricCM16ProstateHCCColon
Visual Feature-level Cosine Similarity43.0055.3373.7255.05
Class-distribution Cosine Similarity41.8162.5573.1259.51
Class-distribution JSD (RePath)47.4367.6275.9860.53

As shown in Table R1, JSD consistently outperforms both cosine similarity variants across all four datasets. Importantly, the strictly controlled comparison confirms that JSD is a core component of RePath's relational affinity formulation, providing consistent improvements independently of the representation space.

We have incorporated this controlled ablation into Sec. 5.3 (p. 8, lines 422–424) and Appendix G.5 (pp. 21-22, lines 1131–1162).

[W2] Clarification of DAI's Contribution Beyond Three-Way Affinity Comparison. DAI is described as a new differential-diagnosis-inspired integration mechanism. However, at the final binary segmentation step, the current formula is mathematically equivalent to checking whether the normalized tumor affinity is the largest of the three classes. The DAI score is positive exactly when the normalized tumor affinity is larger than both the normalized normal affinity and the normalized non-tissue affinity. I suggest that the authors explain what DAI contributes beyond this three-way comparison, or describe it accordingly.

The final binary decision of DAI is mathematically equivalent to a three-way comparison of the normalized tumor, normal, and non-tissue affinities. Importantly, unlike conventional patch-wise VLM classification, which directly compares image–text similarity scores, DAI compares slide-specific relational affinities derived from each patch's correspondence to tumor, normal, and non-tissue anchors (Sec. 4.2). Thus, DAI integrates these relational affinities into a unified segmentation decision inspired by clinical differential diagnosis, rather than introducing a mathematically distinct decision rule.

We have revised Sec. 4.3 (pp. 6-7, lines 321–338) to clarify this equivalence and the role of DAI within RePath's relational inference framework.

[W3] Clarification of Z-Score Normalization and Its Effect on the Decision Boundary. The definition of the z-score normalization is not clear enough. This detail may directly determine whether DAI actually changes the decision boundary relative to a plain argmax over the raw affinities. I suggest that the authors provide pseudocode or write out the full computation.

Shared z-score normalization. We compute shared z-score statistics across the tumor, normal, and non-tissue affinity maps, rather than normalizing each map separately. For each WSI containing NN patches, the shared mean μ\mu and standard deviation σ\sigma are computed over all 3N3N raw affinity scores:

\qquad \sigma = \sqrt{\frac{1}{3N-1}\sum_{m\in\{T,N,J\}}\sum_{i=1}^{N}(R_m(x_i)-\mu)^2}.$$ The normalized affinities are $$\tilde{R}_m(x_i)=\operatorname{sigmoid}\left( \frac{R_m(x_i)-\mu}{\max(\sigma,0.05)+10^{-8}} \right), \quad m\in\{T,N,J\}.$$ Here, the standard deviation is lower-bounded by $0.05$ to prevent excessive amplification of small score variations, and $10^{-8}$ ensures numerical stability. **Effect on the decision boundary.** Shared normalization does not alter the final decision boundary. Since the same strictly increasing transformation is applied to all three affinity maps, their relative ordering is preserved, leaving the final binary segmentation decision unchanged. We have clarified the shared normalization in Sec. 4.2 (p. 6, lines 299–301) and provided the full computation and its effect on the decision boundary in Appendix G.12 (p. 25, lines 1299–1322). > [W4] **Robustness to Incorrect Tumor Anchors.** RePath relies on high-confidence tumor anchors selected from the test slide itself, which depends heavily on the accuracy of the VLM predictions. What happens if the tumor anchors are wrong? For example, under domain shift the VLM may produce false-positive predictions. How would the authors handle this case? The paper includes related experiments, namely tumor anchors drawn from ground-truth regions (Table 4), fully random anchors (Table A10), generic or incorrect keywords (Table A12), and tumor-negative slides (Appendix I). These cover fully correct, fully random, or fully wrong anchors, and wrong keywords. None of them increases the fraction of wrong anchors step by step. I suggest a controlled experiment that gradually introduces mislabeled anchors, to test whether the final results remain robust. We conducted a controlled experiment by randomly replacing 0%, 10%, 20%, 30%, 50%, 70%, and 100% of the selected tumor anchors in $\mathcal{A}_T$ with ground-truth normal patches, while keeping all other components unchanged. **Table R2.** *Robustness to incorrect tumor anchors (IoU, %).* The replacement ratio denotes the percentage of selected tumor anchors in $\mathcal{A}_T$ replaced with ground-truth normal patches. Results are from a single independent run, and thus differ slightly from Table 1. | Replacement Ratio in $\mathcal{A}_T$ (%) | CM16 | Prostate | HCC | Colon | |---|---:|---:|---:|---:| | **0 (RePath)** | 47.4 | 67.6 | 76.0 | 60.5 | | 10 | 48.1 | 67.7 | 75.6 | 60.1 | | 20 | 48.0 | 67.5 | 75.1 | 59.6 | | 30 | 47.6 | 66.5 | 74.2 | 58.7 | | 50 | 42.5 | 59.3 | 71.5 | 55.2 | | 70 | 34.0 | 40.5 | 66.1 | 47.5 | | 100 | 4.0 | 1.9 | 26.9 | 14.7 | RePath maintains relatively stable performance with up to 30% incorrect tumor anchors, with IoU changes within 1.8 percentage points across all four datasets. Performance declines more noticeably at 50% contamination and deteriorates substantially at higher replacement ratios. These results indicate tolerance to moderate anchor contamination, while highlighting the importance of reliable tumor anchor selection under severe prediction errors. We have added this controlled analysis to Appendix G.18 (p. 27, lines 1421–1442).

Dear Reviewer 51P7,

Thank you very much for your time and thoughtful feedback throughout the review process. As the discussion period is approaching its end, we would sincerely appreciate it if you could take a moment to review our responses and share any remaining questions or concerns.

We would be happy to provide further clarification if needed.

Best regards,

The Authors

Reviewer_XMXvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

RePath performs training-free whole-slide tumor segmentation using slide-specific anchors, class-distribution affinity, and differential tumor/normal/non-tissue evidence. The experiments support a useful technical refinement, but the practical need for its zero-shot setting is insufficiently established.

Critical strengths

The multi-organ and multi-backbone evaluation is substantial. The appendix includes useful anchor, propagation, prompt, lesion-size, supervised-baseline, and annotation-scale analyses.

Critical weaknesses
  1. The clinical motivation needs a concrete unmet use case. Existing segmentation models already generalize across cancer types. Skrede et al. [1] report mean Dice above 80% in validation cohorts of en bloc resections and TCGA cohorts, including unseen cancer types. Bedau et al. [2] also demonstrate broad transfer across 21 tumor types, although their evaluation uses selected regions and semiquantitative scoring. These results do not establish that segmentation is solved, but make annotation scarcity alone an insufficient motivation. Which cancer, specimen type, and workflow remain inadequately served by available models? The common prostate, liver, colorectal, and breast-metastasis benchmarks do not themselves establish this gap.

  2. The comparison should establish the value of avoiding target labels. Appendices G.15 and G.16 already compare supervised methods and annotation scales; this evidence should be acknowledged. However, it does not replace comparison with available pretrained cross-cancer segmentation models applied without target adaptation. Evaluate these alternatives on matched held-out cohorts, document source-data overlap, and compare accuracy and manual correction effort against a small annotation budget. This would clarify whether zero-shot inference offers a meaningful practical advantage.

  3. The claimed use case must match the failure boundary. Appendix I reports only 8.6% specificity on negative Camelyon slides, while main comparisons largely condition on tumor-positive slides. The paper also acknowledges failures when backbone predictions miss rare or subtle morphologies, compromising anchor selection. Thus, rare cancers cannot simply justify the method without relevant validation. Distinguish coarse ROI discovery from clinically reliable delineation, and report negative-slide rejection and small-lesion sensitivity for the intended workflow.

  4. The technical increment and decision rule need clarification. CDAM [3] already uses Jensen-Shannon divergence between class distributions. Isolate the metric benefit by comparing cosine and JSD on identical distributions. Specify whether z-score statistics are shared across affinity maps: a common increasing transform cancels when thresholding their difference at zero, whereas separate normalization changes the decision boundary.

References [1] Skrede et al. Generalisation of automatic tumour segmentation in histopathological whole-slide images across multiple cancer types. npj Precision Oncology, 10, 107, 2026. [2] Bedau et al. Comprehensive evaluation of cross cancer generalization in histopathology segmentation models across 21 tumor types. Communications Medicine, 6, 302, 2026. [3] Kang et al. CDAM. ICLR, 2025.

Questions for authors
  1. Task motivation and relevant alternatives: What specific cancer type, specimen type, or clinical workflow motivates the zero-shot setting? Why would existing pretrained cross-cancer segmentation models, applied without target adaptation, be inadequate for this setting? Please clarify how the current benchmarks establish this unmet need.

  2. Intended scope: Is RePath intended only for slides already known to contain tumor, or also for slides with unknown tumor status? Given the 8.6% specificity reported on negative Camelyon slides, how should the main segmentation results be interpreted within the intended workflow?

  3. Inference clarification: Are the z-score statistics shared across the tumor, normal, and non-tissue affinity maps, or computed separately for each map? Please provide the exact normalization and thresholding rule, as this determines whether normalization changes the final segmentation decision.

6 comments

We sincerely thank the reviewer for the thoughtful and constructive feedback. We appreciate the positive assessment of the paper's clarity and soundness, as well as the recognition of our extensive multi-organ and multi-backbone evaluation and supplementary analyses. We address the reviewer's concerns regarding clinical motivation and the practical value of training-free inference below.

[W2] Comparison with Pretrained Cross-Cancer Segmentation Models. The comparison should establish the value of avoiding target labels. Appendices G.15 and G.16 already compare supervised methods and annotation scales; this evidence should be acknowledged. However, it does not replace comparison with available pretrained cross-cancer segmentation models applied without target adaptation. Evaluate these alternatives on matched held-out cohorts, document source-data overlap, and compare accuracy and manual correction effort against a small annotation budget. This would clarify whether zero-shot inference offers a meaningful practical advantage.

We evaluated the pretrained model of Skrede et al. [R1] on the same four WSI benchmarks without target-specific adaptation, and our RePath outperforms it on both unseen organs, while the model of Skrede et al. performs better on the two seen organs (Table R1). Bedau et al. [R2] could not be evaluated because pretrained weights were unavailable. These results indicate that RePath is advantageous for organs outside the training data of pretrained supervised models, whereas such models remain preferable when the target organ is covered by their training data.

Table R1. Comparison with a pretrained cross-cancer segmentation model (IoU, %). Vanilla and RePath results are from Table 1 (CONCH backbone), while Skrede results are from our additional evaluation.

DatasetSkrede Training OrganVanillaRePath (Ours)Skrede et al. [R1]
CM16Unseen33.245.927.1
HCCUnseen63.078.168.2
ProstateSeen51.167.477.4
ColonSeen52.363.064.4

Skrede et al. [R1] trained on four cancer types (colorectal, endometrial, lung, and prostate). Based on the reported cohort descriptions, no source-data overlap was identified with our benchmarks. Evaluating manual correction effort requires a separate pathologist-in-the-loop study beyond the scope of this work.

We have added the Skrede comparison and source-data information to Appendix G.19 (pp. 27-28, lines 1443–1480).

[W1 / Q1] Clinical Motivation and the Role of Zero-Shot Inference.

W1. The clinical motivation needs a concrete unmet use case. Existing segmentation models already generalize across cancer types. Skrede et al. [1] report mean Dice above 80% in validation cohorts of en bloc resections and TCGA cohorts, including unseen cancer types. Bedau et al. [2] also demonstrate broad transfer across 21 tumor types, although their evaluation uses selected regions and semiquantitative scoring. These results do not establish that segmentation is solved, but make annotation scarcity alone an insufficient motivation. Which cancer, specimen type, and workflow remain inadequately served by available models? The common prostate, liver, colorectal, and breast-metastasis benchmarks do not themselves establish this gap.

Q1. Task motivation and relevant alternatives: What specific cancer type, specimen type, or clinical workflow motivates the zero-shot setting? Why would existing pretrained cross-cancer segmentation models, applied without target adaptation, be inadequate for this setting? Please clarify how the current benchmarks establish this unmet need.

A use case for RePath is tumor localization in tumor-bearing WSIs from new organs or institutions without target-specific segmentation training. For cross-organ generalization, our additional comparison (Table R1) shows that RePath outperforms the pretrained model of Skrede et al. [R1] on both unseen organs—Camelyon16 (breast cancer metastases in lymph nodes) and PAIP HCC (liver). For cross-institutional generalization, RePath outperforms the evaluated baselines across all five held-out centers on Camelyon17 (Table 2), without center-specific training.

These results support our slide-specific relational inference as a complementary approach to pretrained pan-cancer segmentation models. Our benchmarks demonstrate comparative performance across organs and institutions, while confirming RePath’s utility in a specific clinical workflow could require additional clinical validation.

We have clarified the clinical motivation and intended scope in the Introduction (p. 1, lines 40–46) and expanded the discussion of pretrained pan-cancer segmentation models in Related Work (Sec. 2.3, p. 3, lines 126–132).

[R1] Skrede et al., Generalisation of automatic tumour segmentation..., npj Precision Oncology, 2026.

[R2] Bedau et al., Comprehensive evaluation of cross cancer generalization..., Communications Medicine, 2026.

[W3 / Q2] Clarification of Intended Clinical Scope and Workflow

W3. The claimed use case must match the failure boundary. Appendix I reports only 8.6% specificity on negative Camelyon slides, while main comparisons largely condition on tumor-positive slides. The paper also acknowledges failures when backbone predictions miss rare or subtle morphologies, compromising anchor selection. Thus, rare cancers cannot simply justify the method without relevant validation. Distinguish coarse ROI discovery from clinically reliable delineation, and report negative-slide rejection and small-lesion sensitivity for the intended workflow.

Q2. Intended scope: Is RePath intended only for slides already known to contain tumor, or also for slides with unknown tumor status? Given the 8.6% specificity reported on negative Camelyon slides, how should the main segmentation results be interpreted within the intended workflow?

Intended scope and workflow. RePath is intended for tumor localization in tumor-bearing WSIs, rather than standalone screening of slides with unknown tumor status. Its 8.6% specificity on tumor-negative Camelyon16 slides (Appendix I) indicates that RePath alone is insufficient for reliable slide-level screening.

A potential approach is to separate slide-level tumor classification from tumor localization. Established methods, including MIL-based classifiers [R3–R4] and zero-shot approaches such as MI-Zero [R5], could provide initial slide-level screening, followed by RePath for tumor segmentation. Accordingly, our main segmentation results should be interpreted as performance on tumor-bearing WSIs, rather than as evidence of end-to-end screening performance.

Small-Lesion Detection and Clinical Limitations. We re-evaluated the tumor-size-stratified analysis on Camelyon16 (Table A12). For small lesions (≤500 tumor patches), RePath achieves 97.1% slide-level detection (34/35 slides) and improves IoU from 17.4% (Vanilla) to 32.7%. These results indicate that RePath can detect small tumor regions, although accurate delineation remains challenging. Rare or subtle morphologies missed by the backbone VLM may still compromise tumor anchor selection.

We have clarified the intended scope in the Introduction (p. 1, lines 44–46), updated the small-lesion analysis in Table A12 (p. 24, lines 1277–1297), and discussed the potential two-stage workflow and clinical limitations in Appendix I (p. 29, lines 1548–1551).

[R3] TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification, NeurIPS, 2021.

[R4] Feature Re-embedding: Towards Foundation Model-Level Performance in Computational Pathology, CVPR, 2024.

[R5] Visual language pretrained multiple instance zero-shot transfer for histopathology images, CVPR, 2023.

[W4 / Q3] Controlled Ablation of JSD vs. Cosine Similarity and Clarification of the Normalization Procedure

W4. The technical increment and decision rule need clarification. CDAM [3] already uses Jensen-Shannon divergence between class distributions. Isolate the metric benefit by comparing cosine and JSD on identical distributions. Specify whether z-score statistics are shared across affinity maps: a common increasing transform cancels when thresholding their difference at zero, whereas separate normalization changes the decision boundary.

Q3. Inference clarification: Are the z-score statistics shared across the tumor, normal, and non-tissue affinity maps, or computed separately for each map? Please provide the exact normalization and thresholding rule, as this determines whether normalization changes the final segmentation decision.

Technical increment over CDAM. While CDAM applies JSD between class distributions to ViT tokens co-present in a single forward pass, RePath addresses inter-patch inference over gigapixel WSIs, where patches are encoded independently. It selects tumor, normal, and non-tissue anchors from the test slide, scores each patch against these slide-specific anchors, and contrasts the resulting category-wise affinities. The controlled ablation below verifies the choice of JSD in this setting rather than claiming it as new.

Controlled comparison of JSD and cosine similarity. To isolate the effect of the similarity metric, we conducted an additional controlled ablation comparing cosine similarity and JSD on identical class distributions, while keeping anchor selection, affinity pooling, and DAI unchanged. Feature-level cosine similarity is also included for reference.

Table R2. Controlled comparison of similarity metrics (IoU, %). All methods use identical anchor selection, affinity pooling, and DAI. The class-distribution cosine similarity and JSD variants differ only in the similarity metric. Results are from a single independent run, and thus differ slightly from Table 1.

Similarity MetricCM16ProstateHCCColon
Feature-level Cosine Similarity43.0055.3373.7255.05
Class-distribution Cosine Similarity41.8162.5573.1259.51
Class-distribution JSD (RePath)47.4367.6275.9860.53

JSD consistently outperforms class-distribution cosine similarity across all four datasets, confirming its importance in RePath's relational affinity formulation independent of representation-space differences. We have added this controlled ablation to Sec. 5.3 (p. 8, lines 422–424) and Appendix G.5 (pp. 21-22, lines 1131–1162).

Shared normalization and decision rule. The z-score statistics are shared across the tumor, normal, and non-tissue affinity maps, not computed separately. For each WSI containing NN patches, the shared mean μ\mu and standard deviation σ\sigma are computed over all 3N3N raw affinity scores:

\qquad \sigma = \sqrt{\frac{1}{3N-1}\sum_{m\in\{T,N,J\}}\sum_{i=1}^{N}(R_m(x_i)-\mu)^2}.$$ The normalized affinities are $$\tilde{R}_m(x_i)=\operatorname{sigmoid}\left( \frac{R_m(x_i)-\mu}{\max(\sigma,0.05)+10^{-8}} \right), \quad m\in\{T,N,J\}.$$ Here, $0.05$ prevents excessive amplification of small score variations, and $10^{-8}$ ensures numerical stability. *Shared normalization does not change the final decision boundary*, as the same strictly increasing transformation preserves the relative ordering of all three affinity maps. Thus, the final binary decision assigns a patch to the tumor class only when its raw tumor affinity exceeds both its normal and non-tissue affinities (Sec. 4.3). We have clarified the shared normalization in Sec. 4.2 (p. 6, lines 299–301), the mathematical equivalence of the decision rule in Sec. 4.3 (pp. 6-7, lines 321–338), and the full normalization procedure in Appendix G.12 (p. 25, lines 1299–1322).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the Area Chair and all reviewers for their efforts and constructive feedback. Before the end of the author–reviewer discussion, we briefly summarize the strengths noted by the reviewers, their main concerns, and how we addressed them, with pointers to the discussion threads.

Strengths noted by the reviewers

  1. A novel motivation and distinctive idea of reformulating independent patch–text classification as relational comparison within the same WSI [51P7], addressing a practical limitation of patch-wise VLM inference with a simple procedure [QJM3].
  2. Significant and consistent gains over zero-shot baselines across all three VLM backbones [51P7, QJM3], exceeding 10-WSI few-shot baselines in average IoU [51P7].
  3. Substantial multi-organ and multi-backbone evaluation with extensive supporting analyses [XMXv, QJM3], including improved prompt robustness with lower variability [QJM3].
  4. Clear presentation [XMXv, 51P7, QJM3] with no substantive concerns about the paper’s soundness [XMXv]

Concerns and our responses

  1. Clinical motivation and comparison with pretrained cross-cancer segmentation models [XMXv W1–W2/Q1]. We clarified the use case as tumor localization in tumor-bearing WSIs from new organs or institutions without target-specific training. RePath outperforms the pretrained pan-cancer model of Skrede et al. on both benchmark organs outside its training data (45.9 vs. 27.1 IoU on Camelyon16; 78.1 vs. 68.2 on PAIP HCC), while, as expected, the supervised model is stronger on organs included in its training. RePath also outperforms all evaluated baselines in IoU on every held-out Camelyon17 center.

  2. Intended scope: tumor-negative slides and small lesions [XMXv W3/Q2; QJM3 W1]. We clarified that RePath is designed for tumor localization within tumor-bearing WSIs, consistent with our main evaluation, and could be paired with existing slide-level classifiers (e.g., MIL-based methods or MI-Zero) for screening. On small lesions, RePath nearly doubles IoU over Vanilla (17.4% → 32.7%).

  3. Novelty relative to CDAM [XMXv W4, follow-up; QJM3 W2]. Our contribution is the inference reformulation: each patch is compared with anchors selected from the same unlabeled slide rather than scored independently against fixed text embeddings. Unlike CDAM's jointly processed ViT tokens, WSI patches are encoded independently, which RePath addresses through slide-specific anchor selection. This training-free reformulation mitigates both prompt dependency and spatial inconsistency, improving segmentation across three backbones and four organs.

  4. Controlled ablation of the similarity metric [51P7 W1; XMXv W4]. We kept anchors, class-distribution representation, pooling, and DAI identical. JSD consistently outperforms both class-distribution and feature-level cosine similarity on all four datasets, by up to 5.6 and 12.3 IoU points, respectively.

  5. DAI decision rule and normalization [51P7 W2–W3; QJM3 W2/Q1; XMXv Q3]. We clarified that DAI integrates slide-specific relational affinities, rather than image–text similarities, into the final decision, which we now describe as a three-way comparison over these affinities. The three affinity maps share z-score statistics, so normalization preserves their ordering.

  6. Robustness to incorrect tumor anchors [51P7 W4]. We progressively replaced tumor anchors with ground-truth normal patches. IoU stays within 1.8 points up to 30% contamination, with larger drops at 50% or higher contamination.

  7. Hyperparameter selection [QJM3 W3/Q2, follow-up]. We repeated the selection on a separate 512-WSI development set with no overlap with the test sets. Both standard and leave-one-dataset-out selection recovered the original confidence threshold. Temperature selection recovered the original values for two of three backbones and, for the third, even improved test IoU by 2.47 points.

  8. Statistical uncertainty and reporting details [QJM3 W3/Q3, follow-up]. Paired slide-level bootstrap 95% confidence intervals exclude zero on all four datasets (mean gains of +8.0 to +24.9 IoU points). We also resolved the reporting inconsistencies in the size-stratified analysis and the anchor-selection timing.

We sincerely thank the AC and all reviewers for helping us clarify the scope and contributions of RePath and strengthen its evaluation. We hope that the rebuttal and revisions address the major concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.