- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method to classify VLMs into the same model family based on their responses to the same VQA questions. Specifically, the CLIP visual and text embeddings for each question's images and responses are extracted and then projected to compute the Euclidean distance between projected vectors. The proposed method is evaluated on diverse base models and achieves high accuracy.
Critical strengths
- The proposed method is simple and straightforward, and achieves high accuracy on diverse models, validating the effectiveness of the proposed method.
- The proposed method proposes a unified representation for multiple tasks, i.e., relation detection, one-to-one verification, and one-to-many base-model identification can be unified within a common representation.
- The paper is well written and the method is easy to follow.
Critical weaknesses
- Novelty limited. The clip based model identification methods have already been explored in previous works. Also the proposed method uses CLIP to construct VQA pair to characterize VLM behavior, the idea shares similarity with previous works.
- The method relies on distances between the behavior differences to the same VQA questions, and assumes models in the same family exhibits similar behavior. However, the data the model finetuned on may also changes the behavior of the model. Does the author studied the influence of the behaviour of models in different family finetuning under the same data.
Questions for authors
Can the author examines the method's effectiveness when models in dfiferent family is finetuned with the same data? And the implact of data aginst the impact of base model.
Dear Reviewer orjm,
Thank you again for your time and effort in reviewing our paper.
As the discussion period draws to a close, we would greatly appreciate any further feedback on our rebuttal and additional experimental results. We remain happy to clarify any outstanding questions or concerns.
Thank you for your consideration.
Best regards,
The Authors