- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this paper, the authors introduce the DICE approach that fuses five frozen pathology foundation models with distinct trainably learnable MIL heads. In the training process, the method utilizes supervised classification, pairwise deep mutual learning, and Gramian volume regularization of projected slide embedding. Predictive entropy based ensemble method is employed for unreliable sample detection, and averaged attention score for localization.
Critical strengths
-
The uncertainty evaluation contains valuable practical checks. The paper considers retained errors as uncertain slides are delayed, applies validation-selected rejection thresholds to test data, and examines both CAMELYON cohort transfer directions. The conclusion reached is that entropy of the ensemble can detect potentially error-prone cases.
-
There is a valuable empirical improvement on the largest multiclass problem. On PANDA, the submission presents F1 score of 76.6% for the full DICE, while the best individual PFM has 72.8%, and early fusion 73.6%. NLL also gets better. The paper uses various frozen encoders and considers feature extraction cost, thus making extra computations evident.
Critical weaknesses
-
The theoretical bound does not lead to uncertainty estimation improvements. Proposition 2 derives from the convexity property of KL divergence and upper-bounds the ensemble Jensen-Shannon disagreement with the KL divergence between two experts. However, making the models agree may also make them confidently wrong about something. Minimizing this measure does not prove better detection of epistemic uncertainty or the error under shift. In addition, the uncertainty estimations do not consider simple late fusion of the same five PFMs, even though this baseline is feasible for classification tasks. Without this comparison, it remains unclear whether the proposed mutual learning improves over heterogeneous ensembling.
-
There is no justification provided for the Gramian penalty's importance and its interpretation. The Gramian penalty is zero if there is any linear dependency among the projected expert vectors. It does not mean that all experts agree on anything or that the diagnostic property of this representation is preserved.
-
No matched heterogeneous ensemble baseline for uncertainty analysis is provided. Late fusion of the same five PFMs without uncertainty analysis is evaluated for classification but is not included in the comparison of uncertainties. In the absence of the risk coverage curve, error detection score, and transferred threshold with matched tuning, the comparison experiments cannot disentangle the effect of mutual learning from the effect of heterogeneous ensembling.
-
The benefits of the entire problem setting are inconsistent across cohorts. With Gramian regularization removed, the improvement of F1 becomes negligible compared to late fusion on PANDA, suggesting that most of the gain comes from the penalty. However, on CAMELYON17, the full DICE obtains an even lower F1 than both without penalty and the best single PFM.
Questions for authors
-
How do typical late fusions of the same five PFMs behave relative to each other on the curves for risk/coverage, error detection measures, validation-selected deferral, and cohort shift?
-
What are the empirical results showing that the smaller volume of the Gramian leads to meaningful alignment and not to redundancy or collapsed alignment?
-
What is the empirical result that directly links the bound on the disagreement term to error detection and not to the overall entropy of the ensemble?
5 comments
Our Rebuttal to Reviewer 9e8J (W2 & Q2 & W4 & Q3)
W2/Q2: Role of the Gramian term
The motivation for the Gramian term is stated in Section 3.2: it introduces representation-level alignment between slide embeddings, complementing the prediction-level coupling provided by DML. Its practical contribution is assessed through our experiments with and without this regularization. We agree that the Gramian penalty vanishes whenever the projected expert vectors are linearly dependent, however, we did not observe the Gramian loss reaching zero during training.
The Gramian term is optimized jointly with supervised classification and DML. The supervised objective encourages each expert to retain information relevant to the diagnostic task, while DML enables knowledge exchange through predictive distributions. Furthermore, the representations being aligned originate from the same slide, providing a shared semantic basis despite differences in the PFMs’ architectures and pretraining. The intended role of Gramian regularization is to encourage alignment across these different representations during task learning.
Finally, we argue that our Gramian objective leads to meaningful alignment, rather than a collapsed one, since DICE outperforms both deep ensembles (one model repeated five times) and late fusion (five-model ensembles) in classification accuracy and uncertainty estimation. We respectfully draw the reviewer’s attention to the fact that this strong empirical performance (Table 1, Figure 2, and Figure 3) shows successful alignment. For example, on PANDA (the most complex and largest dataset in our study), adding Gramian regularization improves F1 from 73.6% to 76.6%, together with improvements in NLL and Brier score.
W4: Clarification on empirical benefits
We respectfully disagree with the reviewer that our empirical benefits are inconsistent. Table 2 reports statistically significant improvements for DICE on PANDA and C16. On C17, however, DICE performs within fold-level variability with the best method, and hence, the slight underperformance is not statistically significant. We would like to draw your attention to the fact that DICE matches or outperforms the aforementioned baselines while also providing better uncertainty estimates.
Q3: Clarification on the theoretical result
Thank you for this question. Our experiments assess error detection using total predictive entropy, while Proposition 2 connects the DML objective to its epistemic component. We do not focus on evaluating a direct relationship between the disagreement term and error detection.
Our choice is motivated by prior medical-classification studies reporting favorable performance of total predictive uncertainty compared with its components or alternative uncertainty measures [1,2]. Furthermore, although the entropy decomposition is mathematically exact, interpreting its components as a faithful separation of aleatoric and epistemic uncertainty has empirically documented limitations [3]. These considerations motivate our focus on total predictive uncertainty for identifying unreliable predictions.
References
[1] Tchokponhoue, G. A. D., & Idri, A. (2026). On the value of uncertainty quantification in deep learning based breast cancer molecular subtype classification. Applied Soft Computing, 186, 114249.
[2] Tchokponhoue, G. A. D., & Idri, A. (2026). A comprehensive empirical evaluation of Bayesian evidential deep learning for uncertainty quantification of image-based medical classification. Results in Engineering, 30, 110453.
[3] Wimmer, L., Sale, Y., Hofman, P., Bischl, B., & Hüllermeier, E. (2023). Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, Proceedings of Machine Learning Research, 216, 2282–2292.
We thank Reviewer 9e8J once again for the time and effort they put into our work. As the discussion period approaches its end, we would kindly like to ask if there are any further comments or concerns that we could address.
Our Rebuttal to Reviewer 9e8J (W1a & W1b & Q1 & W3)
Thank you for reviewing the paper. We appreciate that you find the proposed framework practical and the empirical performance strong. We address your remaining comments below, which we hope will help clarify and improve your evaluation of the paper.
W1a: Relationship between the theoretical analysis and uncertainty estimation
Proposition 2 establishes how the DML objective controls an upper bound on ensemble Jensen–Shannon disagreement through the aggregate pairwise KL divergences between experts. We do not claim that this bound guarantees improved error detection, calibrated disagreement, or protection against confidently incorrect consensus.
The uncertainty measure used in our evaluations is total predictive entropy, which incorporates both the average entropy of individual predictions and their disagreement. Proposition 2 therefore provides a theoretical connection between the training objective and one component of the evaluated uncertainty measure.
Finally, we acknowledge that confidently incorrect consensus is a failure mode that DICE’s uncertainty scores may not detect. However, this failure mode is not specific to our method and exists for all the other reported baselines. In fact, as shown in Figure 2 (right), baseline methods assign misclassified slides uncertainty values comparable to those of their most certain correctly classified slides. This pattern appears less pronounced for both DICE variants, suggesting our method’s superiority.
W1b/Q1/W3: Heterogeneous late-fusion baseline
We agree that including this baseline in the uncertainty analysis is essential to assess the contribution of DML beyond heterogeneous ensembling. We have therefore added Late Fusion to Figures 2 and 3 in the main paper and Figures 6, 7, and 8 in the appendix. We highlighted these changes in blue in the revised manuscript and include the results of Late Fusion, DICE (w/o reg), and DICE in the tables below. These results clearly demonstrate the benefit of our DML loss in uncertainty analysis.
Figure 2, left — PANDA retained error rate (%) ↓
Figure 2, right, and Figure 3(b) — uncertainty separation ↑
All listed separation comparisons have p < 10⁻⁴.
Figure 3(a) — PANDA classification with rejection