- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes TiltMIL, a multiple instance learning method that uses the cumulant-generating function (CGF) to capture different statistical properties of a bag, including higher-order information. The method is evaluated on three pathology datasets, with particularly strong results on PANDA. Ablation studies support the contribution of second-order statistics and consistency regularisation.
Critical strengths
- Interesting approach to bag representation.
Using learned CGF readouts to capture different aspects of a bag's distribution is an interesting idea. The inclusion of second-order statistics is particularly relevant to pathology, where tissue composition can be important.
- Strong experimental results.
The evaluation covers ten baselines and two feature extractors. Results on PANDA are particularly convincing, with TiltMIL achieving 80.73% accuracy compared with 76.31% for the strongest baseline. The ablations support the importance of second-order statistics and consistency regularisation. The robustness experiments are also useful.
Critical weaknesses
- Unclear advantage over simpler pooling methods.
My main concern is that the CGF readouts are closely related to attention-weighted means and variances. It is therefore unclear whether the CGF formulation offers advantages over a simpler multi-query attention model with statistical pooling. A matched comparison would help establish the novelty of the approach.
- Evaluation details and consistency of improvements.
The PANDA results are strong, but improvements on TCGA-NSCLC and TCGA-RCC are much smaller. It would be useful to establish whether these differences are consistent across folds. The authors should also clarify whether cross-validation splits are patient-independent, particularly for TCGA datasets containing multiple slides per patient.
Questions for authors
Could the authors compare TiltMIL with a multi-query attention model using weighted mean and variance pooling under similar training conditions?
Are the cross-validation splits patient-independent, and are the same partitions used for all methods?
Could the authors provide fold-level comparisons for the TCGA datasets to assess whether the smaller improvements are consistent?
2 comments
Response to Reviewer oAAV (2/2)
W2 & Q2 & Q3: Evaluation details and consistency of improvements.
Are the cross-validation splits patient-independent, and are the same partitions used for all methods?
For the TCGA datasets, we strictly enforce patient-independent cross-validation, ensuring that all slides from the same patient remain within the same training, validation, or test partition, thereby preventing patient-level data leakage. For PANDA, patient identifiers are not provided in the original dataset, so we adopt a standard random five-fold split at the slide level. All methods are evaluated using the same data partitions within each experimental setting to ensure fair comparisons.
Could the authors provide fold-level comparisons for the TCGA datasets to assess whether the smaller improvements are consistent?
We thank the reviewer for the suggestion. We conducted paired fold-level comparisons on TCGA-NSCLC and TCGA-RCC using identical patient-independent five-fold partitions. Under ResNet-50 features, TiltMIL improves NSCLC AUC in all five folds, while the other metrics exhibit moderate fold-level variability. Under UNI features, the smaller gains are less consistent: NSCLC Accuracy and Macro-F1 improve in only two of five folds, while RCC shows mixed improvements across metrics. These analyses provide a more detailed assessment of fold-level performance and complement the aggregate results reported in Table 1.
Fold-Level Performance Comparison on TCGA Datasets (ResNet-50).
| Dataset / Metric | Fold 1 Δ | Fold 2 Δ | Fold 3 Δ | Fold 4 Δ | Fold 5 Δ | Mean Δ | Improved Folds |
|---|---|---|---|---|---|---|---|
| NSCLC Acc | -0.91 | +4.55 | +0.47 | -2.46 | +1.49 | +0.63 | 3 |
| NSCLC AUC | +0.78 | +2.03 | +0.32 | +1.64 | +0.43 | +1.04 | 5 |
| NSCLC Macro-F1 | -0.84 | +4.56 | +0.52 | -2.37 | +1.48 | +0.67 | 3 |
| RCC Acc | +0.54 | +0.56 | -3.80 | -5.10 | +4.40 | -0.68 | 3 |
| RCC AUC | +0.07 | +1.97 | -1.47 | -0.55 | +0.95 | +0.19 | 3 |
| RCC Macro-F1 | +1.48 | +0.60 | -4.09 | -3.81 | +6.56 | +0.15 | 3 |
Fold-Level Performance Comparison on TCGA Datasets (UNI).
| Dataset / Metric | Fold 1 Δ | Fold 2 Δ | Fold 3 Δ | Fold 4 Δ | Fold 5 Δ | Mean Δ | Improved Folds |
|---|---|---|---|---|---|---|---|
| NSCLC Acc | -1.37 | +2.02 | -0.47 | -0.49 | +0.50 | +0.04 | 2 |
| NSCLC AUC | -0.42 | -1.11 | -1.07 | -1.41 | -0.62 | -0.93 | 0 |
| NSCLC Macro-F1 | -1.37 | +1.96 | -0.48 | -0.48 | +0.50 | +0.02 | 2 |
| RCC Acc | 0.00 | -1.11 | 0.00 | +2.04 | +1.10 | +0.41 | 2 |
| RCC AUC | -0.07 | -0.50 | -0.28 | +0.10 | +0.01 | -0.15 | 2 |
| RCC Macro-F1 | +0.01 | -0.42 | +0.56 | +2.96 | -0.13 | +0.59 | 3 |
Δ = TiltMIL − best baseline. The fixed reference baselines are ACMIL for NSCLC, 2DMamba for RCC–ResNet-50, and PTCMIL for RCC–UNI.
Response to Reviewer oAAV (1/2)
We thank the reviewer for the effort in reviewing our work. Our point-to-point responses are as follows.
W1 & Q1: On Comparisons with Multi-Query Statistical Pooling
We thank the reviewer for the suggestion. We considered two multi-query attention baselines: Pooling by Multihead Attention (PMA [1]), which uses learnable seed queries to aggregate instance features, and ACMIL-MHA [2], our multi-head attention variant of ACMIL's multi-branch aggregation. For both methods, we extended the original attention-weighted feature aggregation with weighted variance pooling (Var), using the corresponding instance attention weights. Under the same dataset and training protocol, we also examined capacity-matched variants (CM) and the combination of variance pooling and EPC regularization (Var+EPC).
On PANDA–UNI, adding weighted variance improves PMA's Macro-F1 from 70.17% to 71.44%, while further incorporating EPC increases it to 73.13%. Similarly, ACMIL-MHA improves from 67.75% to 69.04% with variance pooling and reaches 71.12% with Var+EPC. Nevertheless, TiltMIL achieves 77.35%, outperforming both extended attention-based baselines. On TCGA-NSCLC–UNI, the differences are smaller. TiltMIL achieves an Accuracy of 93.38% and a Macro-F1 of 93.36%, compared with 92.71% and 92.69% for ACMIL-MHA+Var+EPC, although its AUC is slightly lower (96.94% vs. 97.17%). Furthermore, the original Table 2 shows that removing CGF function values or variance from TiltMIL reduces predictive performance.
These results suggest that augmenting attention-based aggregation with weighted variance and EPC does not fully reproduce the performance of TiltMIL's combined finite statistical readout design, particularly on PANDA.
[1] Set Transformer: A Framework for Attention-Based Permutation-Invariant Neural Networks
[2] Attention-Challenging Multiple Instance Learning for Whole Slide Image Classification