- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a training-free fingerprinting method for identifying model lineage groups. The method selects the top- vocabulary tokens by projecting hidden states at selected depths onto the vocabulary space and compares fingerprints from different models using Jaccard similarity across 250 probes. The experimental results demonstrate the method's effectiveness and robustness.
Critical strengths
- The proposed fingerprinting method is training-free, making it potentially more cost-effective than existing methods. The experiments also demonstrate its robustness under various conditions.
Critical weaknesses
-
Although the experimental results show that fingerprint similarity is consistent with documented model lineage relationships, the comparative evaluation is limited. In particular, the paper does not experimentally compare the proposed method with most of the related methods discussed in the related-work section, making it difficult to assess its performance relative to existing approaches.
-
The paper's presentation becomes less clear starting from Section 3 (Method). The notation (e.g., and in Equation (1)), the explanation of excess similarity in Section 3.3, the rationale for selecting the probe suite, and the explanation of the evaluation metrics (e.g., AUC for lineage-group identification) could be presented more clearly. This makes the methodological foundations of the work difficult to follow and weakens the presentation of its contributions.
Questions for authors
-
Why do you not compare the proposed method with other related methods? Are such comparisons unnecessary, or are direct comparisons difficult because of differences in experimental settings?
-
Have you tested alternative probe-suite selections to assess how sensitive the results are to the choice of probes?