- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper targets an important practical limitation of Pathology Foundation Models (PFMs), whose embeddings encode non-biologically relevant factors, e.g. scanner, source center. To overcome these batch effects, they propose a three-fold approach (HiLO) that leverages a label-aware optimal transport formulation. Specifically, HiLO, first creates two pools of patch embeddings with equal class counts and number of patches per pool. Secondly, they perform the label aware matching of the two sets. Lastly, the method uses the learned map to train an alignment framework jointly with a linear classifier. With this, they aim to overcome the DG objectives' limitations that perform alignment on a distribution rather than a sample level and the label-agnostic nature of the current OT techniques used.
Critical strengths
- The lable-aware extension of the current OT approaches seems well-motivated
- The plug-and-plug nature of the approach that allows it to be combined with other methods, seems flexible and beneficial. The method tackles a really important problem in histopathology and offers an easy deployment solution as it operates on pre-extracted embeddings.
Critical weaknesses
- Methodologically, the extension to label-aware OT feels a bit incremental. Also, the gain from label-awareness over λ_y = 0 is small, which questions the motivation.
- The single proposition provided, while it is true, the proof provided in the appendix does not support the claim, but rather a much weaker version of it.
- The experimental section gives some mixed signals. While the methodology prescribes that , most experiments use . There is no std provided in any of the presented Tables, and given that the method is close to the baselines this is critical. Moreover, the class accuracy drop (Table 9, Appendix) seems large, as scanner accuracy vs class accuracy is not an inherent trade-off of the biological problem but rather the method.
Questions for authors
- While I understand the value of incorporating class labels to the OT formulation, what is the motivation of solving the problem across all tasks C and not per class? As for it holds:
where the subsets that contain the samples of the c class of , respectively and is bijective by definition of the problem. This would enforce the class constraint directly, without the need for and can be computationally more efficient as 1) it requires less memory per class and 2) can be easily parallelized.
-
Re the proof: What the authors prove is that if then the minimizer contains no cross-class match. Is this a typo in proposition's statements? For the stronger version you should have in the spirit of: suppose there is an optimal coupling that minimizes (4) with cost and contains cross-class matches, then . Then if I rematch these pairs to have no cross-class matches, I will have , where the sum is over the new pairs. Since these pairs have matching labels and the feature term in Eq. (3) lies in , we have (strictly negative) for , and so (contradiction). Then, you combine it with the existence of the minimizer with no cross-matches.
-
Why in practice is set to 0.5?
-
Retaining 87% of the class accuracy indicates that this alignment approach, while beneficial for removing scanner accuracy, results in loosing biologically relevant information. How do then other baselines behave in this trade-off, as I understand that Table 9 only concerns HiLO?
-
The method emphasizes scalability but does not report any relevant experiments. What are the wall-clock time and memory costs compared with the other OT baselines?
The authors are grateful for the time and effort spent by the reviewer in assessing our submission, and especially for the corrected argument for Proposition 1. We have addressed all comments by correcting the proof, reporting error bars, and conducting new experiments. Detailed actions and responses to the reviews are as below.
We propose our method HiLO based on the pathological significance and computational cost. Since HiLO allows the cost to stay as a squared Euclidean distance, we can still use the low-rank factorisation of HiRef and scale to WSI datasets. We agree that the gain over λy=0 is small on cleaner labels. However, it is important to note that computational pathology often involves noisy, heterogeneous labels with complex patterns. In our experiments with noisy labels, the gain reaches 0.013 when 30% of the training labels are flipped (Appendix E).
We thank the reviewer for pointing out this mistake. The proof only showed the weaker condition (λy2>2N). We have corrected it with the argument outlined by the reviewer (Appendix A) and acknowledged this in the revised manuscript.
We apologise for the unclear description. Proposition 1 only gives a sufficient condition for the exact solution, and it was not meant as a rule for choosing λy. We have now swept λy from 0 to 2.0 on CONCH v1.5 (Appendix D). OOD macro-F1 is highest at λy=0.5, where the cross-class matches already drop from 18.8% to 0.8%.
We acknowledge that the error bars are required to make a fair comparison. All OOD results are now reported with mean ± std over all combinations of fold and seed, and the claims that HiLO is better on every FM have been removed.
We computed the same probe on ERM features, which come from the same network without alignment (Appendix G). On CONCH v1.5, class accuracy is 0.921 for ERM and 0.879 for HiLO. Section 4.2 is revised accordingly.
We thank the reviewer for this suggestion. We agree that for λy2>2 the problem can be solved per class. We have now added this baseline (Appendix D), which reaches an OOD macro-F1 of 0.811 on CONCH v1.5, compared with 0.817 for HiLO with λy=0.5.
This was a mistake in the proof and not a typo in the statement. Please refer to our response to point 2.
Please refer to our response to point 3.
We have run the same probe for ERM, low-rank OT and Fish on CONCH v1.5 (Appendix G). HiLO gives the lowest scanner accuracy (0.276) and a higher class accuracy than low-rank OT (0.879 against 0.871).
We have added this experiment in Appendix F. At N=32,768, HiLO uses 365 MB of GPU memory, compared with 34.7 GB for Sinkhorn and 17.4 GB for low-rank OT and FRLC. These methods run out of memory on a 40 GB A100 at N=65,536, while HiLO runs up to N=262,144.
Once more, we would like to thank the reviewer for the constructive comments, which have strengthened our paper significantly.