acceptodds
Withdrawn from ICLR 2027

Learning Morphological Evidence Distributions for Whole-Slide Image Classification

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Whole-slide image (WSI) classification based on multiple-instance learning is weakly supervised because each slide contains thousands of tile instances but typically has only a slide-level label. Standard attention pooling only selects informative instances, yet the pooled continuous feature does not by itself identify the recurring morphology that supports or opposes a class. To explicitly identify which recurring morphological patterns support or oppose each diagnosis, we introduce Morphology Evidence Learning via Distributions (MELD), a multiple-instance model in which an attention module determines which tiles to collect information from, a shared tokenizer maps tile features to distributions over morphology tokens, and class-conditional pointwise mutual information (PMI) assigns signed diagnostic evidence to these tokens for both prediction and interpretation. Across four WSI benchmarks and two frozen feature encoders, MELD achieves the best Macro-F1 on six of eight dataset–encoder settings. Pathologist review shows that tiles assigned to the learned morphology tokens consistently exhibit clinically recognizable morphological patterns. Moreover, projecting the PMI evidence back to individual tiles provides localization that outperforms localization based on the attention maps of representative MIL baselines. Finally, MELD can directly guide rate allocation for WSI compression using tumor-class PMI, assigning higher JPEG quality factors to tiles with positive tumor-class evidence and a low quality factor to all remaining tiles, achieving classification performance comparable to that of uniform JPEG compression with up to a \(2.33\times\) reduction in bits per pixel (BPP). Code is available at https://anonymous.4open.science/r/MELD-8F5D/README.mdanonymous.4open.science/MELD.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Whole slide image

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_A3nHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work presents MELD, a weakly supervised approach for whole-slide image classification using attention, morphology tokens, and class-specific diagnostic evidence. The authors evaluated the method on four datasets using two feature encoders. MELD achieved the highest Macro-F1 in six of eight settings. Additional experiments examined tumor localization, morphological consistency, prediction faithfulness, and image compression.

Critical strengths
  1. The work addresses an important limitation of attention-based methods by examining which morphological patterns support or oppose a diagnosis.
  2. The mathematical formulation allows individual tile contributions to be connected directly to the classification score. The deletion experiments provide further support for this connection.
  3. The authors compared the method with 14 baselines using five training seeds. Component ablations and paired bootstrap comparisons strengthen the evaluation.
  4. The pathologist-reviewed examples and compression results demonstrate potential applications of the learned morphology representation.
Critical weaknesses
  1. The methodological novelty should be explained more clearly in relation to existing prototype-based and interpretable MIL methods. Since the shared reference term cancels when comparing class scores, the additional role of PMI beyond prototype classification needs clarification.
  2. The evidence maps used for localization exclude attention weights, whereas the exact prediction contributions include them. Therefore, a region highlighted in the evidence map may have limited influence on the final prediction. The paper should distinguish these two interpretations more clearly.
  3. Different thresholding procedures were used for MELD and the localization baselines. This may affect the comparison. Results using validation-selected thresholds or matched sensitivity would provide stronger evidence of improved localization.
  4. The morphological assessment is mainly qualitative. Although the examples show recognizable tissue patterns, quantitative consistency, agreement between pathologists, and stability across training seeds were not reported.
  5. Some experimental details require clarification, particularly the bootstrap sampling unit and the downstream classifier training procedure used for compression.
Questions for authors
  1.  What does the PMI formulation contribute beyond the empirical prototype classifier, particularly for prediction and interpretation?
    
  2. How do the attention-weighted tile contributions compare with the attention-decoupled evidence maps for localization?
  3. Do the localization improvements remain when thresholds are selected using the same validation procedure for all methods?
  4. Was the pathologist assessment blinded, and how consistently were the sampled tiles assigned the same morphological interpretation?
  5. Were patients or individual slides resampled for the bootstrap analysis? Was DSMIL retrained for each compression setting?
1 comment

Yes, PMI can be extend to the prediction task.

Regarding the thresholds for PMI and attention, the two measures have fundamentally different properties. Attention weights are non-negative and normalized to sum to one, whereas PMI can take both positive and negative values. A positive PMI indicates that a token provides supporting evidence for the class prediction, while a negative PMI indicates evidence against it. Therefore, zero serves as a natural and interpretable threshold for PMI, unlike attention weights.

Reviewer_eR3TOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Morphology Evidence Learning via Distributions (MELD), a weakly supervised framework for whole-slide image (WSI) classification. MELD integrates multi-branch attention over image tiles with a shared morphology tokenizer that maps each tile to a probability distribution over learned codewords. The attention-weighted token assignments are aggregated into a slide-level morphology distribution. Based on class-conditional token prototypes estimated from the training data, the method computes signed pointwise mutual information (PMI) values and uses their expectation under the slide-level distribution to obtain class scores. The authors further establish a connection between this scoring rule and nearest-prototype classification under the Kullback–Leibler (KL) divergence, and provide a decomposition of class scores and pairwise margins into tile-level contributions. The evaluation encompasses four public WSI datasets, two frozen feature encoders, and fourteen MIL baselines, with additional analyses involving five random seeds, paired bootstrap confidence intervals, component and sensitivity ablations, pathologist review of learned tokens, deletion-based faithfulness assessments, tumor localization, and an evidence-guided JPEG compression application.

Critical strengths
  1. Prediction and explanation are coupled by construction. The same attention, token assignment, and signed PMI quantities determine both the slide prediction and the tile-level decomposition (Eqs. 13-16). This is more satisfying than treating an attention map as a post-hoc explanation.
  2. The probabilistic formulation is clear and internally coherent. The slide representation lies on a simplex, PMI gives class-specific positive and negative evidence, and the KL interpretation makes the decision rule easy to inspect. The derivations and stop-gradient prototype-refresh procedure are described in useful detail.
  3. The empirical study is broad and mostly careful. Four datasets, two encoders, controlled reruns of many MIL baselines, five seeds, patient-disjoint splits, a cross-center CAMELYON17 evaluation, and paired bootstrap intervals provide substantially stronger evidence than a single-dataset leaderboard comparison.
  4. The paper evaluates more than classification accuracy. Component ablations, deletion interventions, token galleries, localization experiments, and the independent-DSMIL compression study probe distinct intended roles of the model rather than relying on one aggregate metric.
  5. Presentation quality is high. The main problem, three-module decomposition, notation, training protocol, and relationship between attention and class-specific evidence are generally easy to follow. The limitations section correctly acknowledges the loss of inter-tile spatial organization.
Critical weaknesses
  1. The semantic validity of the learned morphology tokens is supported by a very small and non-blinded qualitative study. For the compact K = 5 BRACS model, one pathologist appears to inspect eight tiles per class-codeword pair, after which four are displayed. The paper does not report the number or experience of reviewers, an annotation protocol, inter-rater reliability, blinding to class/codeword, negative controls, or a quantitative measure of within-token coherence and between-token distinctness. Moreover, this K = 5 demonstration is not the K = 32 model used in the larger visual analyses and may not represent the granularity used for the main results. The current evidence supports that some token groups are visually coherent, but the broader claim that MELD learns stable, clinically recognizable morphology tokens is stronger than the validation.
  2. The interpretability claim needs stronger controls against label-driven association and representation non-identifiability. The tokenizer and codebook are optimized using slide labels, and PMI is computed from class-conditional training distributions. Therefore, a token can be predictive and yield an exact decomposition without corresponding to a unique or stable biological concept. Codeword permutation is harmless, but seed-to-seed splitting, merging, or rotation of morphology partitions would matter for scientific interpretation. The paper reports predictive variance across seeds but not token stability, PMI-rank stability, or whether independently trained models recover comparable morphology. This distinction between computational faithfulness and semantic faithfulness should be explicit.
  3. The localization comparison is not yet fully controlled. Baseline attention maps are min-max rescaled within each slide and thresholded at 0.5, whereas MELD uses a native zero-PMI threshold. These thresholds have different meanings and may favor MELD. The study should compare validation-selected thresholds, threshold-free pixel/tile metrics such as AUPRC or localization AUROC, and uncertainty across seeds. It should also clarify how tile masks are rasterized against pixel annotations and whether tissue-background handling is identical. The impressive Dice and specificity differences are promising, but the current protocol does not isolate signal quality from threshold choice.
  4. Hyperparameter selection and test-set exposure are insufficiently separated. Appendix B states that the validation set is used exclusively for hyperparameter selection, yet Tables 7-9 present many vocabulary-size, temperature, and tokenizer-depth runs on the test sets and discuss which values perform best there. The manuscript does not list the complete validation-selected configuration for every dataset-encoder pair or state whether the main configuration was frozen before these test sweeps. Repeated test-set inspection can lead to optimistic method development even if each individual model is selected by validation performance. The authors should document the search space, selection criterion, chosen value per setting, and whether sensitivity tables were produced only after the main configuration was locked.
  5. The compression application omits an important systems cost and deployment assumption. Evidence-guided allocation first requires MELD/UNI processing of the uncompressed WSI to determine which tiles receive a higher JPEG quality factor. This can be reasonable for offline archival or re-encoding, but it is not a free encoder-side signal. The claimed 2.33x BPP reduction appears to count only stored JPEG bits and not the cost of initial high-quality acquisition, feature extraction, the evidence map/side information, or compute. The paper should define the intended use case and report encoding-time overhead and all metadata needed to reconstruct the adaptively compressed slide.
  6. The incremental source of the predictive gain is somewhat difficult to attribute. The KL/PMI equivalence is an elegant algebraic interpretation of empirical class prototypes rather than a new optimization principle by itself. The full method also combines multi-branch gated attention, diversity regularization, a learned tokenizer, and a prototype classifier. The ablations are useful, but the most relevant conceptual comparison would include a matched non-tokenized KL/prototype classifier and stronger controls for whether the gain comes from discretization, class prototypes, or extra attention capacity. This would sharpen the novelty claim relative to prototype- and concept-based MIL.
Questions for authors
  1. Please provide the complete hyperparameter search and selection record for each dataset-encoder setting: candidate values, validation criterion, selected K, assignment temperature, tokenizer depth, branch count, diversity weight, and whether the main configuration was frozen before test-set sensitivity sweeps in Tables 7-9.
  2. Can the authors report seed-to-seed stability of learned codewords and class-token PMI structure, for example via matched codewords, assignment agreement, or stability of the top positive/negative morphology groups? How often do the same pathologic patterns emerge in independently trained models?
  3. Please re-evaluate localization with thresholds selected on validation data for every method and with at least one threshold-free metric, reporting variability across seeds. Also clarify tile-to-pixel rasterization, tissue masking, and whether all baselines use the same slide preprocessing and feature encoder.
  4. Please describe the pathologist study protocol in enough detail to assess semantic validity: reviewer count and expertise, blinding, sampling frame, class/codeword information shown to the reviewer, agreement criteria, and whether the K = 5 findings hold for the K = 32 models used elsewhere.
  5. For evidence-guided compression, what information and computation are available before compression, what side information must be stored, and which costs are included in BPP? Please state the intended deployment scenario and report encoder-side runtime or compute overhead relative to uniform JPEG
1 comment

Thank you for your positive comments.

Regarding the thresholds for PMI and attention, the two measures have fundamentally different properties. Attention weights are non-negative and normalized to sum to one, whereas PMI can take both positive and negative values. A positive PMI indicates that a token provides supporting evidence for the class prediction, while a negative PMI indicates evidence against it. Therefore, zero serves as a natural and interpretable threshold for PMI, unlike attention weights.

Regarding evidence-guided compression, we use JPEG to compress WSI patches, following the DICOM-supported compression framework. Since JPEG is self-contained and embeds the quantization tables within the compressed file, no additional side information is required for decoding.

Reviewer_e7jyOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The manuscript presents MELD, a MIL framework designed to make explicit which repeated morphological patterns support or contradict a WSI diagnosis. It integrates multi-branch attention with a shared tokenizer to present every slide as a distribution over learned morphology tokens, next employs class-specific PMI evidence for prediction and interpretation. The authors demonstrate that this classification rule is equivalent to comparing distributions via KL divergence and that its scores and class margins can be decomposed into tile-level contributions. They evaluate MELD via classification experiments, morphology inspection, deletion tests, localization, and evidence-guided compression.

Critical strengths
  1. The model offers a clear link between token representation, class-specific evidence, and prediction. The KL and decomposition identities are valid, and the deletion experiments supplement algebraic reconstruction by recomputing the complete model after intervention.
  2. The experiments are thorough, spanning four datasets and two frozen encoders, including 14 re-run baselines and five random seeds.
Critical weaknesses
  1. The main morphology-interpretability contribution requires stronger validation and explanation. Current SSL features already capture valuable morphology; MELD's unique idea is to arrange it into clear, diagnostically informative tokens. The pathologist-reviewed galleries are relevant, but their consistency, held-out provenance, and stability are not quantitatively demonstrated against current representations. There are additionally particular problems in the supporting figures: Figure 3 repeats CW5 tiles across malignant and atypical panels, and Figure 4 seems inconsistent with the normalization constraints of the stated class-balanced PMI. These need reconciliation before the galleries can reliably support the central claim. Moreover, the attention-decoupled score rsiyr^y_{si} used for localization differs from the contribution τsiy\tau^y_{si} covered by the decomposition and deletion analysis (lines 289–305, 440–454, and 1029–1047).
  2. The classification benefit is limited and inconsistent across settings. Using ViT-S features, CAM16 Macro-AUROC increases by just 0.003 over the strongest baseline, whereas CAM17 Macro-AUROC is 0.896 versus ACMIL's 0.906. Though greater gains occur elsewhere, these results make convincing evidence of the added interpretability value particularly important for establishing the method's practical significance.
  3. The comparisons do not yet demonstrate the advantage over simple pooling and class-specific explanation options. Tables 3–6 contain uniform weighting, which is a valuable mean-aggregation control within MELD, but Table 1 lacks standard max/mean pooling on matched features. Earlier studies [1] provide such informative controls, especially for patch localization; published scores should not be compared directly across different protocols. The localization evaluation compares MELD's zero-threshold signed evidence against attention maps rescaled per slide and thresholded at 0.5, without signed attribution or instance-classifier baselines. This supports superiority over those attention-map rules, but leaves the broader interpretability advantage uncertain.
  4. When I examined the linked code repository on October 17, it was empty. This limits reproducibility since the manuscript does not completely document preprocessing, per-setting model configurations, and hyperparameter-search budgets.

References [1] Jafarinia et al., “Navigating the MIL Trade-Off: Flexible Pooling for Whole Slide Image Classification”.

Questions for authors
  1. Could you provide the raw prototypes/PMI values and plotting transformation for Figure 4? Equations (11)–(12) imply ∑yexp⁡(eyb[k])=C\sum_y \exp(e_{yb}[k])=C, and Eq. (17) implies ∑yexp⁡(eˉy[k])≤C\sum_y \exp(\bar e_y[k])\le C. For BRACS, two entries for one token cannot both surpass log⁡(3/2)\log(3/2), while the two positive CW2 cells seem to do so. Please additionally explain why Figure 3's malignant CW5 rows 2–3 reoccur as atypical CW5 rows 1–2, and provide their slide/coordinate provenance.
  2. With matched encoders, splits, and tuning, how does MELD compare with conventional max/mean pooling and class-specific explanations such as Additive MIL, xMIL, or instance-classifier scores? For localization, validation-selected thresholds or threshold-free metrics could help distinguish signal quality from thresholding. Could you additionally measure token coherence or stability relative to a representation built from the same frozen features?
  3. What annotated CAM17 ITC slides and patients contribute to CAM17 results, and are they exclusively held out? Please specify the localization aggregation protocol, per-class classification results, and whether the bootstrap resamples patients or individual slides. Accessible split files and full per-setting configurations would help answer such questions; I could not independently access the linked repository.
  4. As UNI is already used in the compression experiments, can the authors add a matched classification comparison using UNI features in Table 1? This could help assess whether MELD's benefits persist with stronger pathology representations.
  5. Please provide the implementation, executable configurations, and exact data partitions needed to reproduce the reported results.

One small mathematical correction: Corollary 1 establishes monotonicity in every distance gap with the others constant. The sentence following Eq. (34) should not imply that shared-parameter optimization automatically improves every competing-class gap.

1 comment

The repository is accessible; please double-check the link. We acknowledge the typo in the figure, which was introduced during the draw.io export process. We appreciate the reviewer’s careful examination and attention to detail.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.