- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes the Persistence Ratio (PR), a transferability estimator based on zero-dimensional persistent homology of pretrained embeddings. PR characterizes cluster connectivity to rank candidate models before fine-tuning. Experiments on CNNs and ViTs report higher average correlations with downstream performance than the evaluated baselines, with source-domain PR performing better for CNNs and target-domain PR performing better for ViTs.
Critical strengths
- The scoring rule is simple and requires neither fine-tuning nor sample-level labels, making it potentially useful when annotations or computational resources are limited.
- The comparison between source- and target-domain topology offers an interesting perspective on representation structure under domain shift.
Critical weaknesses
-
Missing comparisons. The evaluation omits TTP [1], despite identifying it as the closest prior work, and relevant label-free baselines such as RankMe [2] and TranSAC [3]. Comparisons with competitive estimators including DISCO [4], LEAD [5], and ITM [6] would also help establish PR’s relative merits. These comparisons should distinguish supervision requirements, access to source data, and computational costs. Without them, the reported improvements provide limited evidence of competitiveness.
-
Limited evaluation. Evaluating CNNs and ViTs separately does not establish whether PR can reliably rank them within a shared candidate pool. A more informative evaluation would jointly include different architectures and pretraining paradigms, such as supervised learning, contrastive learning, and masked image modeling. ITM [6] provides a relevant reference protocol. This setting is particularly important because the preferred scoring domain differs between CNNs and ViTs in the reported experiments.
-
Unclear link to adaptability. PR measures unlabeled connectivity, but well-separated clusters need not align with the semantic distinctions required by the target task. Moreover, architectures and pretraining paradigms can exhibit different adaptation dynamics [6], motivating methods that model fine-tuning evolution [5,6]. Whether PR’s static score reliably predicts performance across these settings remains unclear. Although the memorization experiment illustrates one failure case, stronger controlled analyses on naturally pretrained models are needed to establish the method’s reliability and limitations.
[1] TTP: On Characterizing the Evolution of Embedding Space of Neural Networks Using Algebraic Topology. PRL, 2024.
[2] RankMe: RankMe: Assessing the Downstream Performance of Pretrained Self-Supervised Representations by Their Rank. ICML, 2023.
[3] TranSAC: TranSAC: An Unsupervised Transferability Metric Based on Task Speciality and Domain Commonality. Pattern Recognition, 2026.
[4] DISCO: Assessing Pre-Trained Models for Transfer Learning Through Distribution of Spectral Components. AAAI, 2025.
[5] LEAD: LEAD: Exploring Logit Space Evolution for Model Selection. CVPR, 2024.
[6] ITM: Implicit Modeling for Transferability Estimation of Vision Foundation Models. NeurIPS, 2025.
Questions for authors
-
What assumptions connect the unlabeled cluster structure measured by PR to the semantic distinctions required by the downstream task? What target-task information enters source-domain PR, and how does it distinguish suitability for different target tasks?
-
How does PR perform on a heterogeneous benchmark, such as the model pool evaluated in ITM [6], under matched fine-tuning protocols? Please report per-dataset ranking correlations, including weighted Kendall’s τ, alongside overall averages and scoring costs.
-
Does PR remain predictive across models with different adaptation dynamics, particularly contrastively pretrained and masked-image-modeling models? How should the scoring domain be selected for a mixed model pool without consulting downstream fine-tuning results?