- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes FFM-CP, a few-shot fusion framework for pathology vision-language foundation models. It aligns frozen encoders' embeddings using support-set-derived Orthogonal Procrustes maps, then jointly refines support-image features and visual and textual class prototypes through a multi-relation graph. Predictions combine text-prototype and case-retrieval streams across ordered backbone pairs. Experiments evaluate three model combinations on six patch/ROI classification datasets at 4, 8, and 16 shots per class.
Critical strengths
-
The paper addresses a useful problem: exploiting complementary pathology VLMs when labeled examples are scarce. Its closed-form Procrustes alignment matches heterogeneous embedding spaces while preserving within-model geometry, and the unified graph jointly refines support-image features and visual and textual class prototypes across backbones. Cross-backbone pair fusion allows queries encoded by one model to draw on evidence represented by another. Together, these components provide a coherent approach to using cross-model information during both adaptation and prediction.
-
The full framework achieves higher mean macro-F1 than its strongest constituent in 50 of 54 comparisons. In Table 4, Procrustes also achieves higher mean macro-F1 than zero-padding and the learned MLP mapping on all six datasets.
Critical weaknesses
-
The experiments do not yet establish whether the full design justifies its added complexity. All Table 4 variants retain graph refinement, so its contribution is not isolated, and the diagonal variant still uses the joint graph. A straightforward ensemble of independently adapted models is missing. The ablation covers only one model combination at 16 shots and provides no paired statistical analysis. The computational and memory costs of using three VLMs are not quantified.
-
Reproducibility and backbone selection require clarification. The prediction and fusion computations and basic training settings are insufficiently specified. For datasets other than BRACS, the use of original splits and patient/slide separation for any new partitions is unclear. The three fusion configurations are reported without an explained selection rule, and the evaluation omits CONCH. These gaps limit confidence in the interpretation and reproducibility of the gains.
Questions for authors
Please check the Critical Weaknesses section first.
-
The current comparisons do not isolate graph refinement or establish an advantage over simple ensembling. Table 4 also covers only one backbone combination at 16 shots. Please include a controlled graph ablation and an ensemble of independently adapted models under the same few-shot protocol, extend the ablation to representative combinations and shot settings, and provide paired uncertainty estimates or statistical analysis for the differences in Table 4.
-
The rationale and data used to select the three fusion configurations are unclear. Please report the selection criteria and any other combinations explored, assess sensitivity to backbone composition, and explain how to select models for a new few-shot task. Please also include CONCH [1] as an individual baseline and in a representative fusion configuration, or explain its exclusion.
-
The prediction procedure, training settings, and computational cost are insufficiently documented. Please provide an algorithm table with training and inference pseudocode, specify how branch scores are computed and fused, and report the optimizer, learning rate, and number of epochs. Report feature-precomputation and adaptation times, end-to-end inference time, and peak memory use relative to the strongest individual model and a simple ensemble under comparable hardware and settings. Please also correct the duplicated wording immediately before Equation (2), on page 4, line 184.
-
For datasets other than BRACS, the use of original splits is unclear. Please state whether each dataset's original training, validation, and test partitions were used where available. For any created or modified splits, explain whether and how patient- and slide-level separation was enforced to prevent related tiles or ROIs from appearing across partitions.
-
The evaluation is limited to tile/ROI classification with image-text VLMs. Please discuss the modifications needed for slide-level inference, vision-only encoders, and broader multimodal pathology backbones. As further validation, include a representative slide-level analysis, using MI-Zero [2] as an example of MIL-based VLM aggregation, to assess whether fusion gains extend to whole-slide classification.
[1] Lu, Ming Y., et al. "A visual-language foundation model for computational pathology." Nature medicine 30.3 (2024): 863-874. [2] Lu, Ming Y., et al. "Visual language pretrained multiple instance zero-shot transfer for histopathology images." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.