acceptodds
Under review as a conference paper at ICLR 2027

Hyperbolic Vision-Language Model for Pathological Patch-Slide Representation

Abstract

Gigapixel whole-slide image (WSI) analysis requires capturing localized pathological evidence at the patch level and integrating these fine-grained cues into a slide-level representation. Although slide-level foundation models automate holistic tissue analysis, their Euclidean representations discard feature-norm information during normalization and patch aggregation, even though this information could help organize hierarchical relationships and retain clinically meaningful fine-grained spatial cues. In this work, we propose HYPERION, a hyperbolic vision-language model for pathological patch-slide representation that leverages the inherent norm structure of pretrained Euclidean foundation models. HYPERION constructs text-conditioned Einstein midpoints over patch embeddings to summarize the patches most similar to each caption and uses entailment cones to enforce hierarchical inclusion among caption, midpoint, and slide representations. Confidence-filtered caption-patch correspondences provide pseudo-labeled patch-level contrastive supervision from slide-level normal or abnormal labels, while intermediate feature norms are propagated to the radii of final patch embeddings to expose norm information. In zero-shot tumor segmentation, HYPERION has the highest Dice on five benchmarks, while prediction evaluations show that HYPERION preserves or enhances pathological knowledge. Integrated into a WSI multimodal large language model, HYPERION improves overall report-generation metrics and yields more lesion-focused visual grounding in qualitative analyses, verifying the utility of hyperbolic patch-slide representations for pathology.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Histopathological Slide Representation, Hyperbolic Embedding, Grounding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MEY1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HYPERION fine-tunes TITAN in Lorentz space using caption-conditioned Einstein midpoints, contrastive and entailment losses, confidence-filtered patch pseudo-labels, and intermediate-norm propagation. It greatly improves zero-shot tumor localization, roughly preserves slide-level prediction, and gives small report-generation gains with qualitatively better MLLM lesion grounding.

Critical strengths

The paper tests whether the gain harms the original representation, using 36 slide-level tasks and an MLLM. This is much broader than segmentation alone.

Critical weaknesses

The key mechanistic claim is under-isolated: patch pseudo-supervision is the largest contributor, the matched Euclidean ablation is under-specified, and no radius/cone-activity diagnostics establish that the model is genuinely exploiting hyperbolic geometry rather than mainly improved supervision and radial rescaling. In addition, cross-model segmentation Dice uses different native tile grids and excludes negative slides, so the absolute “best on five benchmarks” comparison is less conclusive than the strong same-grid HYPERION-vs-TITAN result.

Questions for authors
  1. Could the authors precisely define “w/o hyperbolic space”? Report matched Euclidean-vs-hyperbolic controls with radius, cone-activation, and hierarchy-violation statistics, plus variance across seeds for Tab. 4.

  2. Can all methods be evaluated on a common grid, with false-positive rates on tumor-negative slides?

  3. Are mutation captions used in L_pat? How accurate are the pseudo-abnormal and pseudo-normal labels against ground truth, given that low similarity to one abnormal caption need not imply normal tissue?

6 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer MEY1 [1/3]

We thank the reviewer for recognizing HYPERION’s improved zero-shot tumor localization and MLLM lesion grounding, and our evaluation across 36 slide-level tasks and an MLLM to assess whether grounding gains preserve the original representations. We address the reviewer’s comments below.

[W1, Q1] The key mechanistic claim is under-isolated: patch pseudo-supervision is the largest contributor, the matched Euclidean ablation is under-specified, and no radius/cone-activity diagnostics establish that the model is genuinely exploiting hyperbolic geometry rather than mainly improved supervision and radial rescaling. Table 4 does not report variance across seeds.

We address four points: 1) the contribution beyond patch alignment, 2) clarification of the “w/o hyperbolic space” and matched comparisons with unnormalized Euclidean learning, 3) the interpretation of the radius and cone diagnostics, and 4) variance across seeds for Table 4.

1) Patch-level alignment does not account for HYPERION’s full improvement. Table R1 shows that hyperbolic learning provides a clearly favorable structure for grounding, further strengthened by patch alignment, by comparing diverse fine-tuning configurations under matched data and budget. Specifically, Euclidean slide alignment achieves overall Dice of 0.219, below TITAN’s 0.263. Adding pseudo-caption-based patch alignment raises Dice to 0.458, still below HYPERION (0.606). Without patch alignment, HYPERION reaches 0.399, substantially outperforming TITAN. Along the Lorentz sequence, Dice increases to 0.345 with entailment, 0.366 with midpoint learning, and 0.399 with norm propagation, without patch supervision.

Table R1. Fine-tuning TITAN with the same training data and budget across four datasets (Dice). Line in bold is HYPERION (ours)

SpaceEntail.MidpointNorm prop.Patch align.CM16CM17PAIP ProstatePAIP ColonOverall
Euc.––––0.1430.0980.2480.5630.263
Euc.✗✗✗✗0.1730.1120.3330.2600.219
Euc.✗✗✗✓0.3420.2550.6300.6070.458
Lorentz✗✗✗✗0.1660.1170.3740.5670.306
Lorentz✗✗✗✓0.3240.2740.6560.7400.499
Lorentz✓✗✗✗0.2730.2070.2080.6920.345
Lorentz✗✓✗✗0.1840.1360.4260.5770.331
Lorentz✗✗✓✗0.2070.1570.4460.6170.357
Lorentz✓✓✗✗0.2950.2240.2780.6680.366
Lorentz✓✓✓✗0.2960.2380.4210.6410.399
Lorentz✓✓✓✓0.4630.4040.7730.7830.606

We included the matched fine-tuning controls in Sec. 5.5 (L. 430–431, 460–462) and the final rows of Table 4 (L. 456–458), with the incremental analysis and per-dataset results in Appendix C.3 (L. 1239–1246) and Table A14, and the single-component Lorentz controls in Table 5.

2) Clarification of the “w/o hyperbolic space” ablation and matched comparison to unnormalized Euclidean controls. The “w/o hyperbolic space” in Table 4 means replacing Lorentz representations with unnormalized Euclidean ones while retaining all other proposed components. We note that Table R1 uses standard (L2-normalized) contrastive fine-tuning as in TITAN, whereas Table R2 uses unnormalized Euclidean controls; hence their Euclidean entries differ (0.219 vs. 0.266 with slide alignment; 0.458 vs. 0.455 with patch alignment). By extending this control to matched ablations in both spaces, we found that Lorentz-space learning consistently outperforms unnormalized Euclidean learning (Table R2). Specifically, Lorentz learning outperforms its Euclidean counterpart across all evaluated configurations and datasets. Notably, adding the remaining components to patch alignment improves average Dice from 0.455 to 0.501 in Euclidean space (+0.046), versus 0.499 to 0.606 in Lorentz space (+0.107). This larger gain supports the benefit of hyperbolic hierarchical modeling beyond norm preservation alone.

Table R2. Component ablations in unnormalized Euclidean (Euc.) and Lorentz (Lor.) spaces.

Entail.MidpointNorm prop.Patch align.CM16 (Euc./Lor.)CM17 (Euc./Lor.)PAIP Prostate (Euc./Lor.)PAIP Colon (Euc./Lor.)Average (Euc./Lor.)
✗✗✗✗0.122/0.1660.068/0.1170.345/0.3740.527/0.5670.266/0.306
✓✗✗✗0.114/0.2730.066/0.2070.120/0.2080.325/0.6920.156/0.345
✗✓✗✗0.129/0.1840.072/0.1360.348/0.4260.569/0.5770.280/0.331
✗✗✓✗0.125/0.2070.070/0.1570.366/0.4460.525/0.6170.272/0.357
✗✗✗✓0.307/0.3240.240/0.2740.595/0.6560.679/0.7400.455/0.499
✓✓✓✓0.271/0.4630.286/0.4040.700/0.7730.746/0.7830.501/0.606

We clarified the definition of “w/o hyperbolic space” in Sec. 5.5 (L. 421–422, 466–467) and Appendix A.7 (L. 998–1003), and added the matched Euclidean–Lorentz comparison to Sec. 5.5 (L. 464–470) and Table 5.

AuthorsOpenReview ↗

Rebuttal for Reviewer MEY1 [2/3]

3) Radius and cone diagnostics support the intended ordering. We added HoroPCA projections of held-out TCGA slides to Appendix C.4 (Figure A3), comparing TITAN, HYPERION, and ablations without hyperbolic space or norm propagation. Original-space radii confirm r(t)<r(μt)<r(s)r(t)<r(\mu_t)<r(s) on all slides (0.76, 1.23, 2.03), with pseudo-tumor/normal radii of 1.55/1.27. Before fine-tuning, the midpoint lies inside the caption radius (0.36 vs. 0.78), and groups overlap in projection. Without hyperbolic space, the midpoint radius collapses to 0.0055 (caption: 0.38; slide: 0.88).

Table R3 reports cone-apex angles (φ\varphi) for the three learned containment relations on TCGA slides across five seeds. Smaller angles indicate tighter alignment of a child with its parent’s cone axis. Fine-tuning reduces TITAN’s caption-to-slide and caption-to-midpoint angles from 132.6° and 151.8° to 35.5° and 39.7° for HYPERION. The ablations further show how the hierarchy changes when model components are removed.

Table R3. Entailment cone angle statistics on held-out TCGA.

Methodφt→s\varphi_{t\to s}φt→μt\varphi_{t\to\mu_t}φμt→s\varphi_{\mu_t\to s}
TITAN (before fine-tuning)132.6151.888.9
HYPERION35.5±1.339.7±0.940.4±2.6
w/o hyperbolic space73.8±4.6165.7±0.938.1±2.3
w/o entailment65.0±5.163.3±6.372.3±3.1
w/o midpoint contrastive46.8±2.351.0±2.856.8±1.2
w/o LpatL_{\mathrm{pat}}68.9±4.187.7±7.638.2±2.0
w/o norm propagation42.3±2.147.9±1.853.5±1.3

We included the HoroPCA projections and radius statistics in Appendix C.4 (L. 1295–1305) and Figure A3, and the cone-angle statistics in Appendix C.4 (L. 1306–1313) and Table A16, referenced in Sec. 5.5 (L. 469).

4) The variance across seeds for Table 4. Table R4 shows the standard deviations for Table 4. We included the standard deviations in Appendix C.3 (L. 1259–1262) and Table A15.

Table R4. Component ablation of Table 4 over five seeds (Dice, mean ± std).

SettingCM16CM17PAIP ProstatePAIP ColonOverall
TITAN (no fine-tuning)0.1430.0980.2480.5630.263
HYPERION0.463±0.0230.404±0.0280.773±0.0040.783±0.0340.606±0.022
w/o hyperbolic space0.271±0.0490.286±0.0150.700±0.0290.746±0.0200.501±0.016
w/o entailment0.364±0.0450.306±0.0210.734±0.0200.701±0.0090.526±0.012
w/o midpoint contrastive0.408±0.0310.375±0.0300.689±0.0190.688±0.0290.540±0.021
w/o LpatL_{pat}0.296±0.0180.238±0.0130.421±0.0350.641±0.0240.399±0.018
w/o norm propagation0.330±0.0200.256±0.0240.613±0.0340.756±0.0150.489±0.016

[W2] In addition, cross-model segmentation Dice uses different native tile grids and excludes negative slides, so the absolute “best on five benchmarks” comparison is less conclusive than the strong same-grid HYPERION-vs-TITAN result.

We evaluated all methods on a common grid in both directions: on the 224 and 512 grid (Table R5). Consequently, HYPERION remains the best on every common grid, so the ranking does not depend on the grid. Every method uses the identical grid and ground-truth rasterization. We exclude PRISM because it is the only method without per-tile outputs. We already reported PRISM in its native block setting in Table 1. From Table R5, HYPERION achieves the best Dice in all eight settings. Its margin over the strongest baseline is 0.13–0.19 on CAMELYON16, CAMELYON17, and PAIP Colon, and about 0.35 on PAIP Prostate. The ranking does not depend on the choice of grid. We address the negative slides in our response to Q2.

Table R5. Dice on common grids (224 grid / 512 grid).

Dice (224 grid / 512 grid)CM16CM17PAIP ProstatePAIP Colon
HYPERION (Ours)0.416 / 0.4630.381 / 0.4040.759 / 0.7730.778 / 0.783
PathGen-CLIP0.265 / 0.2820.250 / 0.2510.263 / 0.2830.586 / 0.607
CHIEF0.146 / 0.1730.085 / 0.0960.402 / 0.4350.571 / 0.600
CONCH0.155 / 0.1830.112 / 0.1380.291 / 0.3030.503 / 0.604
TITAN0.128 / 0.1430.090 / 0.0980.235 / 0.2480.564 / 0.563

We clarified the grid settings in Appendix A.2 (L. 862–863), and included the common-grid comparisons in Appendix D.1 (L. 1403, 1428–1433) and Table A20.

AuthorsOpenReview ↗

Rebuttal for Reviewer MEY1 [3/3]

[Q2] Can all methods be evaluated on a common grid, with false-positive rates on tumor-negative slides?

We measured tile-level false positives on tumor-free CM16 and CM17 slides, reporting the median and zero-FP slide fraction alongside the outlier-sensitive mean (Table R6). HYPERION keeps false positives near zero without sacrificing tumor detection. CONCH, CHIEF, TITAN, and PRISM flag 23–52% of normal tissue as tumor, exceeding 1% on 95–100% of slides. HYPERION maintains near-zero FPR on both grids (mean 0.6% on CM16, 0.06% on CM17), with the most zero-FP slides (32.1% and 53.5%).

Table R6. False-positive-rates (FPR) on tumor-negative slides (224 grid / 512 grid). † PRISM has no per-tile output, so it is reported only on its native 224 grid.

MethodMean FPR ↓Median FPR ↓FP=0 slides ↑FP>1% slides ↓
CAMELYON16 normal
HYPERION0.0058/0.00550.00047/0.0004632.1%/32.1%1.2%/1.2%
PathGen-CLIP0.0263/0.00490.01764/0.001260.0%/17.3%86.4%/12.3%
CONCH0.3397/0.32350.32678/0.291220.0%/0.0%100%/100%
CHIEF0.4761/0.45030.47193/0.448660.0%/0.0%100%/100%
TITAN0.4926/0.48810.50413/0.496750.0%/0.0%100%/100%
PRISM†0.2991/–0.27276/–0.0%/–100%/–
CAMELYON17 negative
HYPERION0.0006/0.00060.00000/0.0000053.5%/53.5%0.3%/0.3%
PathGen-CLIP0.0152/0.00200.01254/0.001110.3%/21.1%61.6%/2.2%
CONCH0.2988/0.25800.26730/0.222570.0%/0.0%100%/100%
CHIEF0.5158/0.49110.52408/0.495680.0%/0.3%99.7%/99.7%
TITAN0.3994/0.39610.39900/0.394260.0%/0.0%99.7%/99.7%
PRISM†0.2273/–0.14609/–0.6%/–95.3%/–

We clarified the evaluation on tumor-negative slides in Appendix A.1 (L. 782–783), and included the false-positive analysis in Appendix D.1 (L. 1444–1453) and Table A21.

[Q3] Are mutation captions used in L_pat? How accurate are the pseudo-abnormal and pseudo-normal labels against ground truth, given that low similarity to one abnormal caption need not imply normal tissue?

Mutation captions. Yes, each slide carries one unified caption pool formed as the union of its report captions and its mutation captions. We clarified the unified pool of report and mutation captions in Appendix A.4 (L. 951–953).

Pseudo-label precision and tumor burden. We found pseudo-normal precision of at least 98.9% across tumor-burden groups and high pseudo-abnormal precision in ranges relevant to training (Tables R7–R9). As TCGA lacks patch captions, we used tumor annotations from CAMELYON16, CAMELYON17, PAIP Prostate, and PAIP Colon. We pooled patches by slide tumor-overlap fraction. AA denotes patches paired with abnormal captions in Lpat\mathcal{L}_{\text{pat}}, and NN the caption-specific pseudo-normal set.

Tumor contamination in NN was low across all groups (Table R7), despite the ambiguity of inferring normality from low caption similarity. For tumor burden ≥1%, average AA precision was 90.0%, 94.1%, 79.2%, and 72.3%, respectively (Table R8). Precision was limited below 1%, occurring only in CAMELYON16 and CAMELYON17. However, such cases represented only 0.4% of training slides, while 99.2% had ≥25% tumor cells (Table R9), suggesting limited training impact. TCGA tumor-cell percentages came from the portal metadata field percent_tumor_cells.

Table R7. Pseudo-normal precision stratified by tumor burden.

Benchmark / Tumor burden<1%1–5%5–10%10–25%25–50%≥50%
CAMELYON16100% (2,258/2,258)100% (1,143/1,143)100% (1,264/1,264)100% (717/717)100% (93/93)100% (27/27)
CAMELYON1799.93% (1,472/1,473)100% (978/978)100% (644/644)100% (298/298)100% (172/172)—
PAIP Prostate—99.93% (4,536/4,539)99.92% (3,639/3,642)100% (5,182/5,182)99.35% (1,676/1,687)98.90% (450/455)
PAIP Colon—100% (407/407)100% (1,624/1,624)100% (4,226/4,226)99.97% (5,903/5,905)99.23% (2,196/2,213)

Table R8. Pseudo-abnormal precision stratified by tumor burden.

Benchmark / Tumor burden<1%1–5%5–10%10–25%25–50%≥50%
CAMELYON1610.3% (73/711)65.0% (169/260)99.6% (557/559)91.4% (469/513)89.7% (87/97)93.7% (118/126)
CAMELYON176.0% (42/702)98.6% (273/277)92.2% (178/193)100% (120/120)71.8% (51/71)—
PAIP Prostate—79.0% (828/1,048)86.6% (1,248/1,441)82.5% (2,123/2,572)65.3% (611/936)53.5% (131/245)
PAIP Colon—13.2% (14/106)44.9% (241/537)68.8% (1,826/2,653)78.3% (2,591/3,307)78.7% (1,221/1,551)

Table R9. Tumor-cell percentage of the TCGA slides used for training.

Metric/Tumor cells<1%1–5%5–10%10–25%25–50%≥50%50–75%75–90%≥90%
Slides2602254086,1151,6071,8442,664
Share0.4%0.0%<0.1%0.4%6.2%93.0%24.4%28.0%40.5%

We added the pseudo-label precision and tumor-burden analyses to Appendix C.5 (L. 1377–1393) and Tables A18–A19.

Reviewer_WKD8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces HYPERION, a framework maps representations of slices, patches, and text into the Lorentz hyperbolic space. First, it calculates text-conditioned Einstein midpoints based on the similarity between patches and text, thereby constructing local visual evidence for each textual description. Second, it establishes a hierarchical relationship, linking "textual concepts" to "local evidence" and subsequently to "specific patches", through contrastive learning and entailment cone constraints. Furthermore, it selects pseudo-anomalous and pseudo-normal patches from anomalous slices and, in conjunction with normal slices, provides patch-level supervision. Finally, it incorporates the norm of intermediate Transformer features into the hyperbolic radius of the final patch representations, allowing magnitude information, present prior to normalization, to contribute to the learning process. The proposed algorithm has been evaluated across five segmentation datasets, 36 slice-level prediction tasks, and a report generation task, consistently demonstrating strong performance.

Critical strengths

(1) This paper uses hyperbolic space to model the patch, slide, and text features of pathology, which is a very interesting perspective, and the design is quite sound.

(2) This paper provides extensive validation across multiple tasks, consistently achieving strong results, the experiments and supporting details are very comprehensive.

Critical weaknesses

(1) The existing ablation studies fail to demonstrate that the performance gains stem specifically from hyperbolic hierarchical modeling, rather than from fine-tuning where the norm is preserved.

The paper posits that the "discarding of norms" in Euclidean representations is a primary motivation. However, the loss of norm information is a consequence of specific normalization techniques and similarity metric designs, it is not an inherent limitation of Euclidean space itself. Euclidean models that retain magnitude could similarly leverage norm information.

While the authors do provide a baseline that removes the hyperbolic space component, this variant is not sufficiently detailed. For instance, it remains unclear whether this baseline retains the same mechanisms for norm propagation, text-conditioned aggregation, and the use of patch-level pseudo-labels.

Furthermore, the paper lacks visualizations of the hyperbolic features, a common feature in research on hyperbolic representation learning, which would help verify whether a hierarchical structure has actually been learned.

(2) There is a critical interpretational issue regarding the evidence for the norm-based mechanism: an AUC of 0 does not imply the absence of discriminative information. In Figure 6, the authors compare a scenario where norm propagation is disabled (resulting in a norm-based AUC of 0.00) against their full proposed method, arguing that norm propagation facilitates the formation of discriminative cues for lesions. However, by definition, an AUC near 0 indicates a near-perfect inverse ranking, rather than a lack of discriminative capability.

(3) Treating "mismatch with a given anomaly description" as "normal" introduces a semantic leap in the supervision strategy. In Equation (8), the authors select patches with low similarity to the anomaly-describing text as "pseudo-normal" samples, which are subsequently paired with "normal" text in Equation (9). The issue here is that a single image slice may contain multiple types of anomalies; therefore, one cannot simply assume that a low similarity score (i.e., 1−θ1-\theta) equates to a "normal" sample.

(4) The most significant contribution of this paper lies in its application across diverse task types. However, from an algorithmic perspective, the level of novelty may be somewhat limited. Existing literature already includes various approaches that utilize hyperbolic space to model pathological features, as well as pathology-related text and spatial transcriptomics data. This paper adapts the "slide foundation model" framework, extending it to classification and segmentation tasks, and even validating it on multimodal large language models. The volume of work is substantial, and I truly appreciate the authors' efforts.

Questions for authors

See details in Weakness.

5 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer WKD8 [1/2]

We thank the reviewer for recognizing our approach to modeling pathology patch, slide, and text features in hyperbolic space as an interesting and sound perspective. We appreciate the recognition of our extensive evaluation, consistently strong results, and complete experimental details. We address the reviewer’s comments below.

[W1] The existing ablation studies fail to demonstrate that the performance gains stem specifically from hyperbolic hierarchical modeling, rather than from fine-tuning where the norm is preserved. ... Furthermore, the paper lacks visualizations of the hyperbolic features, a common feature in research on hyperbolic representation learning, which would help verify whether a hierarchical structure has actually been learned.

We address the reviewer's concerns in two parts: 1) comparisons against unnormalized Euclidean learning to examine the contribution of hyperbolic modeling, and 2) visualization of the hierarchical structure of the learned representations.

1. Hyperbolic modeling beyond norm preservation. We revised our motivation to focus on normalization in contrastive learning, which removes feature magnitude, rather than Euclidean geometry itself. We compared HYPERION’s components in unnormalized Euclidean and Lorentz spaces in Table R1. The Euclidean configurations retain feature magnitudes, to assess whether norm preservation alone accounts for the improvement. Across all configurations and datasets, Lorentz-space learning outperforms its unnormalized Euclidean counterpart. Adding entailment, midpoint, and norm propagation to patch alignment increases Dice from 0.455 to 0.501 (= “w/o hyperbolic space” in Table 4) in Euclidean space (+0.046), compared with 0.499 to 0.606 in Lorentz space (+0.107). These components provide a larger gain within the hyperbolic framework, supporting the benefits of hyperbolic hierarchical modeling beyond simply retaining norm information.

Table R1. Component ablations in unnormalized Euclidean (Euc.) and Lorentz (Lor.) spaces.

Entail.MidpointNorm prop.Patch align.CM16 (Euc./Lor.)CM17 (Euc./Lor.)PAIP Prostate (Euc./Lor.)PAIP Colon (Euc./Lor.)Average
✗✗✗✗0.122/0.1660.068/0.1170.345/0.3740.527/0.5670.266/0.306
✓✗✗✗0.114/0.2730.066/0.2070.120/0.2080.325/0.6920.156/0.345
✗✓✗✗0.129/0.1840.072/0.1360.348/0.4260.569/0.5770.280/0.331
✗✗✓✗0.125/0.2070.070/0.1570.366/0.4460.525/0.6170.272/0.357
✗✗✗✓0.307/0.3240.240/0.2740.595/0.6560.679/0.7400.455/0.499
✓✓✓✓0.271/0.4630.286/0.4040.700/0.7730.746/0.7830.501/0.606

We revised the motivation (L. 14–19, 50–53, 100–102, 464–465), added Euclidean–Lorentz controls (L. 465–470; Table 5).

2. Visualization of the learned hierarchical structure. We added HoroPCA projections of TCGA slides to Appendix C.4 (Figure A3), comparing TITAN, HYPERION, and ablations without hyperbolic space or norm propagation. HYPERION satisfies r(t)<r(μt)<r(s)r(t)<r(\mu_t)<r(s) on all slides (0.76, 1.23, 2.03); pseudo-tumor/normal radii are 1.55/1.27. Before fine-tuning, the midpoint lies inside the caption radius (0.36 vs. 0.78), with group overlap. Without hyperbolic space, the midpoint collapses to 0.0055, versus caption/slide radii of 0.38/0.88, supporting the role of hyperbolic geometry.

We added plots to Appendix C.4, Fig. A4, comparing slide-wise z-scored patch-embedding norm distributions for tumor and normal tiles across four annotated benchmarks, using TITAN before and HYPERION after fine-tuning. On qualifying slides, HYPERION has greater tumor-normal mean separation (Δμ\Delta\mu) across all benchmarks: 1.44 vs. 0.51 on CAMELYON16, 1.32 vs. 1.16 on CAMELYON17, 1.10 vs. 0.93 on PAIP Colon, and 1.24 vs. 0.84 on PAIP Prostate.

We included the hierarchy visualizations and tumor–normal norm distributions in Appendix C.4 (L. 1295–1349), Figures A3–A4, with references in Sec. 5.5 (L. 469) and Appendix C.1 (L. 1141–1142).

[W2] There is a critical interpretational issue regarding the evidence for the norm-based mechanism: an AUC of 0 does not imply the absence of discriminative information. In Figure 6, the authors compare a scenario where norm propagation is disabled (resulting in a norm-based AUC of 0.00) against their full proposed method, arguing that norm propagation facilitates the formation of discriminative cues for lesions. However, by definition, an AUC near 0 indicates a near-perfect inverse ranking, rather than a lack of discriminative capability.

We thank the reviewer for clarifying the interpretation of an AUC near zero. Norm propagation is intended to align the radial ordering with HYPERION’s hierarchy, so we revised the Discussion accordingly (Sec. 6, L. 481–485 and 498–499) to describe AUC 0.00 as an inverse ranking relative to this hierarchy following the reviewer’s comments.

AuthorsOpenReview ↗

Rebuttal for Reviewer WKD8 [2/2]

[W3] Treating "mismatch with a given anomaly description" as "normal" introduces a semantic leap in the supervision strategy. In Equation (8), the authors select patches with low similarity to the anomaly-describing text as "pseudo-normal" samples, which are subsequently paired with "normal" text in Equation (9). The issue here is that a single image slice may contain multiple types of anomalies; therefore, one cannot simply assume that a low similarity score (i.e., 1−θ1-\theta) equates to a "normal" sample.

Following the reviewer’s comments, we found low tumor contamination in caption-specific pseudo-normal sets, although low similarity to one anomaly description need not imply normal tissue (Table R2). We compared N(t)N(t) with a conservative normal reference Nall=⋂tN(t)N_{\text{all}}=\bigcap_t N(t), defined using all abnormal captions for each organ as patches responding to none. We measured coverage of NallN_{\text{all}} by N(t)N(t) and tumor fractions in N(t)N(t), NallN_{\text{all}}, and N(t)∖NallN(t)\setminus N_{\text{all}}. The last set responds to other captions, directly addressing the concern. We analyzed TCGA, our training distribution, and benchmarks with pixel-level tumor annotations.

Three observations address the concern. First, N(t)N(t) covers 98.7% of NallN_{\text{all}} on TCGA and 96.8–100% on annotated benchmarks. Second, N(t)N(t) contains at most 2.6% tumor across annotated benchmarks, and 0.00% on CAMELYON16. Third, the suspect region N(t)∖NallN(t)\setminus N_{\text{all}} contains 0.00% tumor on CAMELYON16 and CAMELYON17, and 0.36% and 1.96% on PAIP Prostate and Colon. Given the difficulty of exhaustive tumor annotation in large histopathology datasets, this low contamination supports the utility of pseudo-normal supervision without dense tumor labels during training. Although the semantic ambiguity remains, its measured impact on the pseudo-normal set appears limited.

Table R2. Tumor contamination in caption-specific pseudo-normal sets and coverage of NallN_{\text{all}} across benchmarks. TCGA has no pixel-level annotations, so tumor fractions are reported only for the annotated benchmarks

BenchmarkSlidesTumor rate∣N(t)∖Nall∣∣N(t)∣\frac{\lvert N(t)\setminus N_{\text{all}}\rvert}{\lvert N(t)\rvert}Tumor in N(t)N(t)Tumor in N(t)∖NallN(t)\setminus N_{\text{all}}Slides with no tumor in N(t)N(t)NallN_{\text{all}} coverage by N(t)N(t)
TCGA train (training dist.)*299—58.2%———98.7%
CAMELYON16487.7%55.7%0.00%0.00%48/4896.8%
CAMELYON17493.8%53.6%0.07%0.00%48/4999.5%
PAIP Prostate20019.2%46.7%0.34%0.36%195/200100.0%
PAIP Colon20036.7%52.9%2.62%1.96%194/20097.1%

We added the coverage and tumor-contamination analysis to Appendix C.5 (L. 1356–1365) and Table A17.

[W4] The most significant contribution of this paper lies in its application across diverse task types. However, from an algorithmic perspective, the level of novelty may be somewhat limited. Existing literature already includes various approaches that utilize hyperbolic space to model pathological features, as well as pathology-related text and spatial transcriptomics data. This paper adapts the "slide foundation model" framework, extending it to classification and segmentation tasks, and even validating it on multimodal large language models. The volume of work is substantial, and I truly appreciate the authors' efforts.

We thank the reviewer for recognizing the breadth of our evaluation. Beyond extending hyperbolic learning to a slide foundation model, HYPERION introduces three methodological advances that, to our knowledge, are novel relative to prior hyperbolic representation learning methods in pathology or natural-image domains.

First, HYPERION uses a text-conditioned Einstein midpoint to aggregate caption-relevant patches and incorporate the resulting local evidence into both alignment and entailment objectives. This formulation has not been explored in existing hyperbolic representation learning methods for pathology such as HyperPath (Huang et al., 2025) and HyperST (Zhang et al., 2026a).

Second, whereas HyCoCLIP (Pal et al., 2025) and UNCHA (Kim et al., 2026) on natural images require ground-truth region–text pairs, HYPERION learns patch-level alignment through pseudo-captions without ground-truth patch–text correspondences, avoiding costly patch-level caption annotation in gigapixel WSIs while improving grounding.

Third, norm propagation transfers pretrained feature norms to hyperbolic radii despite architectural normalization, a mechanism unexplored in prior hyperbolic representation learning methods. Our ablations support these methodological contributions, demonstrating algorithmic novelty alongside the breadth of applications recognized by the reviewer.

AuthorsOpenReview ↗

A gentle note on our response

Dear Reviewer,

Thank you again for your thoughtful review and helpful comments on our paper.

We just wanted to share a brief note on our response. Following your comments, we compared each component in unnormalized Euclidean and Lorentz spaces to separate the effect of hyperbolic modeling from norm preservation (Table R1), added visualizations of the learned hierarchy (Appendix C.4), revised our interpretation of the AUC result of norm propagation, analyzed tumor contamination in the pseudo-normal sets (Table R2), and clarified the methodological novelty.

We understand this is a very busy time, and we sincerely appreciate the time and effort you have already devoted to reviewing our paper. If time permits before the discussion period closes, we would be grateful if you could read through our response. Any further thoughts or feedback would be very welcome, and we would be happy to clarify anything.

Best regards, The Authors

Reviewer_LNEBOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HYPERION, a hyperbolic fine-tuning framework for pathology vision-language models that aims to strengthen semantic correspondence between local patches and slide-level text while preserving whole-slide representation capabilities. Built upon TITAN, the method aggregates relevant local evidence through text-conditioned Einstein midpoints, uses entailment constraints to establish a semantic hierarchy among text, local evidence, and slides, and optimizes representations through confidence-filtered patch-level pseudo-label supervision and intermediate feature norm propagation. Experiments demonstrate substantial improvements in zero-shot patch-level tumor segmentation, along with some gains in slide-level classification and report generation.

Critical strengths
  1. The paper is well structured, with a coherent methodological design. Motivated by the pathological information encoded in pretrained feature norms, the paper introduces hyperbolic space into pathology foundation model fine-tuning. Text-conditioned aggregation, hierarchical constraints, and local alignment form a coherent progression from motivation to method.
  2. The experimental evaluation is broad. The paper evaluates tumor segmentation, slide-level classification, and report generation, examining both local lesion localization and global semantic representation and generation capabilities. Ablation studies further assess the contributions of individual components.
Critical weaknesses
  1. The fairness of the comparisons is questionable. HYPERION receives explicit patch-level alignment during fine-tuning, whereas the baseline foundation models are evaluated without comparable additional training. Consequently, its superior segmentation performance is unsurprising and does not, by itself, establish the advantage of the proposed method under matched training conditions.
  2. The source of the performance gains remains insufficiently isolated. The method uses additional textual supervision, which I understand is necessary for the overall framework. However, this makes it difficult to determine whether the improvements arise from the proposed methodological design or from the additional supervision. Although the paper includes an ablation without hyperbolic space, this does not isolate the contribution of additional textual supervision relative to the original foundation model.
Questions for authors

Please address the concerns raised in the Critical Weaknesses, particularly the fairness of the comparisons and the attribution of the performance gains.

5 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer LNEB [1/2]

We thank the reviewer for the insightful comments and for recognizing the paper’s clear structure, coherent progression from motivation to method, and broad evaluation across tumor segmentation, slide-level classification, and report generation. We address the reviewer’s comments below.

[W1] The fairness of the comparisons is questionable. HYPERION receives explicit patch-level alignment during fine-tuning, whereas the baseline foundation models are evaluated without comparable additional training. Consequently, its superior segmentation performance is unsurprising and does not, by itself, establish the advantage of the proposed method under matched training conditions.

We respectfully emphasize that HYPERION’s grounding gains cannot be attributed solely to patch-level alignment, which is itself one of our methodological contributions, while we agree that comparisons with foundation models without additional training may not isolate our contribution. To address reviewer’s concerns, we show that (1) existing patch-level alignment does not guarantee spatially coherent grounding, and (2) our fine-tuning controls distinguish the gains of patch-level alignment from those of our hyperbolic representation framework, as shown in Table R1.

Table R1. Segmentation Dice across four datasets. Overall is the mean across datasets. Space: Euclidean (Euc.) or Lorentz (Lor.)

MethodSpaceCM16CM17PAIP ProstatePAIP ColonOverall
PathGen-CLIP (from Table 1)–0.2650.2500.2630.5860.341
PRISM (from Table 1)–0.2310.1560.2840.5210.298
TITAN (from Table 1)–0.1430.0980.2480.5630.263
Fine-tuning TITAN without LpatL_{pat}Euc.0.1730.1120.3330.2600.219
Fine-tuning TITAN with LpatL_{pat}Euc.0.3420.2550.6300.6070.458
HYPERION without LpatL_{pat} (from Table 4)Lor.0.2960.2380.4210.6410.399
HYPERION (Ours)Lor.0.4630.4040.7730.7830.606

1) Existing patch-level alignment does not guarantee spatially coherent grounding. Patch-level encoders such as CONCH and PathGen-CLIP already learn patch–text alignment. As shown in Table R1, PathGen-CLIP achieves stronger segmentation performance than the existing slide encoders, confirming the value of patch-level supervision. However, these patch encoders process patches independently and therefore do not explicitly incorporate neighboring tissue context into their representations. As illustrated in Figs. A5–A8, adjacent patches belonging to the same tissue can receive conflicting predictions, resulting in spatially inconsistent and noisy segmentation. Thus, learning patch–text correspondence alone does not resolve the grounding problem addressed by HYPERION.

2) Our fine-tuning controls distinguish the benefit of patch-level alignment from that of the proposed hyperbolic representation framework. Table R1 compares fine-tuning configurations starting from TITAN with the same training data and budget. We can see that Euclidean fine-tuning without pseudo-caption-based patch alignment (LpatL_{pat}) does not improve grounding, while adding this improves segmentation. Nevertheless, it still underperforms HYPERION by a large margin (0.458 vs. 0.606), indicating that patch alignment alone does not account for the full improvement. Our ablations provide complementary evidence (Table 4 in the main). Even without LpatL_{pat}, HYPERION clearly improves grounding over the initial TITAN model (0.399 vs. 0.263), supporting the contribution of its hierarchical representation learning, orthogonal to explicit patch alignment, and then adding LpatL_{pat} within the complete HYPERION framework further improves performance. Therefore, attributing the entire gain only to patch-level alignment overlooks the contribution of the proposed hyperbolic representation learning framework.

In the revision, we added the fine-tuned TITAN controls to Sec. 5.5 (L. 430–431, 460–462) and Table 4 (L. 456–458), with the full incremental analysis in Appendix C.3 (L. 1239–1246) and Table A14.

AuthorsOpenReview ↗

Rebuttal for Reviewer LNEB [2/2]

[W2] The source of the performance gains remains insufficiently isolated. The method uses additional textual supervision, which I understand is necessary for the overall framework. However, this makes it difficult to determine whether the improvements arise from the proposed methodological design or from the additional supervision. Although the paper includes an ablation without hyperbolic space, this does not isolate the contribution of additional textual supervision relative to the original foundation model.

To distinguish the contribution of our methodological design from additional textual supervision, we fine-tune TITAN using the same training data and budget under different configurations. Additional textual supervision alone does not explain the gains, which emerge only when the proposed components are combined (Table R2). We first evaluate Euclidean fine-tuning with slide alignment alone and with additional patch alignment (already shown in Table R1). We then examine Lorentz-space fine-tuning with (i) slide alignment alone, (ii) each HYPERION component added individually, and (iii) incremental incorporation of the components.

As shown in Table R2, slide alignment alone does not consistently improve grounding: Euclidean overall Dice drops from 0.263 to 0.219, particularly on PAIP Colon (0.563 to 0.260). Patch alignment raises overall Dice to 0.458, still below HYPERION (0.606). Lorentz controls outperform Euclidean ones both without patch alignment (0.306 vs. 0.219) and with it (0.499 vs. 0.458). Adding entailment, Einstein midpoint, or norm propagation individually improves over the Lorentz slide-alignment baseline. Incrementally, Dice rises to 0.345 with entailment, 0.366 with midpoint learning, 0.399 with norm propagation, and 0.606 with patch alignment. These controls show that additional text alone does not explain the gains; the full improvement comes from combining the proposed representation learning components.

Table R2. Dice across the four datasets after fine-tuning TITAN with the same training data and budget: Euclidean (Euc.) and Lorentz (Lor.) configurations with slide alignment alone, individual additions of HYPERION components, and their incremental combination.

MethodSpaceEntail.MidpointNorm prop.Patch align.CM16CM17PAIP ProstatePAIP ColonOverall
TITAN (no fine-tuning)Euc.––––0.1430.0980.2480.5630.263
TITAN fine-tuningEuc.✗✗✗✗0.1730.1120.3330.2600.219
TITAN fine-tuningEuc.✗✗✗✓0.3420.2550.6300.6070.458
TITAN fine-tuningLor.✗✗✗✗0.1660.1170.3740.5670.306
TITAN fine-tuningLor.✗✗✗✓0.3240.2740.6560.7400.499
TITAN fine-tuningLor.✓✗✗✗0.2730.2070.2080.6920.345
TITAN fine-tuningLor.✗✓✗✗0.1840.1360.4260.5770.331
TITAN fine-tuningLor.✗✗✓✗0.2070.1570.4460.6170.357
TITAN fine-tuningLor.✓✓✗✗0.2950.2240.2780.6680.366
TITAN fine-tuningLor.✓✓✓✗0.2960.2380.4210.6410.399
HYPERION (Ours)Lor.✓✓✓✓0.4630.4040.7730.7830.606

We added TITAN controls (L. 430–431, 460–462; Table 4, L. 456–458), incremental and per-dataset analyses including PAIP Colon degradation (L. 1239–1246; Table A14), and individual component additions (Table 5, Lorentz columns).

AuthorsOpenReview ↗

A gentle note on our response

Dear Reviewer,

Thank you again for your thoughtful review and helpful comments on our paper.

We just wanted to share a brief note on our response. Following your comments, we added matched fine-tuning controls on TITAN with the same training data and budget (Tables R1–R2), showing that neither patch-level alignment nor additional textual supervision alone accounts for the gains.

We understand this is a very busy time, and we sincerely appreciate the time and effort you have already devoted to reviewing our paper. If time permits before the discussion period closes, we would be grateful if you could read through our response. Any further thoughts or feedback would be very welcome, and we would be happy to clarify anything.

Best regards, The Authors

Comments on the paper

2 comments
Area_Chair_4hofOpenReview ↗

A Gentle Nudge: Let's Keep the Discussion Going!

Dear Reviewers,

The authors have now posted their responses, addressing your concerns and incorporating your suggestions. We are already halfway through the discussion period!

One of the distinguishing features of ICLR is its interactive review process, where constructive discussions not only help strengthen individual papers but also contribute valuable insights to the broader scientific community.

So, let's not let the discussion be a one-way street! I encourage you to revisit the authors' responses, acknowledge their efforts, and engage in further discussion where needed. After all, a good scientific discussion is often as valuable as the final verdict!

-AC

AuthorsOpenReview ↗

General Response

We thank the Area Chair and all reviewers for their efforts and constructive feedback. Before the end of the author-reviewer discussion, we briefly summarize the reviewers' noted strengths, their concerns, and how we addressed them, with pointers to the threads.

Strengths noted by the reviewers

  1. An interesting and sound perspective of modeling patch, slide, and text in hyperbolic space, with a coherent progression from motivation to method [WKD8, LNEB].
  2. Broad evaluation across segmentation, classification, and report generation, with ablations [LNEB, WKD8, MEY1]; testing whether the gains harm the original representation is broader than segmentation alone [MEY1].
  3. Large gains in zero-shot tumor localization while preserving slide-level prediction [MEY1, LNEB].
  4. Clear presentation [LNEB, MEY1].

Concerns and our responses

  1. Novelty relative to prior hyperbolic methods [WKD8 W4; LNEB follow-up]. We clarified three differences: (i) unlike prior hyperbolic pathology methods, HYPERION improves grounding via caption-conditioned Einstein midpoints within a slide foundation model; (ii) it learns patch-level alignment from pseudo-captions derived from slide-level labels, rather than the ground-truth region–text pairs required by other hyperbolic VLMs; and (iii) it introduces a novel norm propagation, unexplored in either domain, that exposes pre-normalization magnitudes as hyperbolic radii.

  2. Attribution of gains: matched training vs. additional supervision [LNEB W1–W2; MEY1 W1/Q1]. We added fine-tuning controls on TITAN with the same data and budget, showing that additional supervision alone does not explain the gains: slide alignment lowers Dice (0.263 → 0.219), adding patch alignment reaches 0.458 vs. HYPERION's 0.606, and HYPERION without patch alignment still reaches 0.399.

  3. Hyperbolic geometry vs. norm preservation; definition of "w/o hyperbolic space" [WKD8 W1; MEY1 W1/Q1]. We clarified that this ablation uses unnormalized Euclidean space with all other components retained, and added matched unnormalized-Euclidean vs. Lorentz controls for each component. As a result, Lorentz was better in every configuration and dataset, indicating gains beyond norm preservation. We also revised the motivation to attribute norm loss to normalization rather than Euclidean geometry.

  4. Evidence that the hierarchy is learned; seed variance [WKD8 W1; MEY1 Q1]. We added HoroPCA visualizations confirming the intended radial order (caption < midpoint < slide), five-seed cone-angle statistics, and five-seed standard deviations for the component ablation, which preserve its ranking. Following the reviewers' follow-ups, we clarified that the patch ordering is an observed outcome rather than an enforced constraint, and that cone angles indicate tighter hierarchical alignment by the entailment loss as a soft penalty.

  5. Interpretation of AUC ≈ 0 in Fig. 6 [WKD8 W2, follow-up (3)]. We corrected our interpretation that an AUC near zero indicates a near-perfect inverse ranking, not a lack of discriminative information, and revised the text accordingly. This AUC measures whether patches the model selects as pseudo-abnormal have larger norms than pseudo-normal ones, rather than localization against ground truth. We showed that norm propagation aligns the representation ordering of the pretrained model into the intended direction, consistent with the radial hierarchy HYPERION aims to learn.

  6. Pseudo-normal supervision: low similarity to one abnormal caption need not imply normal tissue [WKD8 W3; MEY1 Q3; LNEB follow-up]. We quantified its impact on annotated benchmarks to see the effectiveness of our pseudo-normal supervision, while we acknowledged that this ambiguity could still be remained. Caption-specific pseudo-normal sets contain ≤2.62% tumor with ≥98.9% precision across tumor-burden groups, and pseudo-abnormal precision is limited only on very low-burden slides, which are rare in our training data.

  7. Segmentation protocol: native tile grids and tumor-negative slides [MEY1 W2/Q2]. We evaluated all methods on common 224 and 512 grids, including PRISM and PANDA, and HYPERION remained the best in all settings. We also reported false-positive rates on tumor-negative slides, where HYPERION keeps false positives near zero.

We sincerely thank the AC and the reviewers for their time and efforts throughout the review period. We hope that our rebuttal and revisions have relieved the major concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.