- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a method called delta activation to identify whether (or how) a foundation model is fine-tuned by datasets on specific domains. Delta activation uses the difference of hidden states of models before and after fine-tuning to identify the changes during fine-tuning, and uses the delta as the fingerprint of the fine-tuning process. For the delta activation, the authors perform clustering experiments to show the effectiveness of the fingerprint. Users can cluster the fine-tuned model to specific domain and know how the model is fine-tuned.
Critical strengths
The paper falls in an interesting scope that use the parameters or hidden states rather than model outputs as the "fingerprint" or "watermark" of LLMs. This method might be helpful as an supplementary way to identify the authorship of LLMs or fine-tuning.
The paper is easy to follow and has clear statements.
Critical weaknesses
The motivation of the paper is the main concern. I personally do not see any use case of the propose scenario: selecting model by clustering the activations/hidden states. For LLM using, people either use API or fine-tuned model now. For APIs, there is no need and not possible to have the hidden states. For open-sourced fine-tuned models, why there is a need to know what dataset is used to fine-tune the model? Why we do not know what dataset or task the model is fine-tuned on? If not, why we are using the model?
The paper seems not ready to be published. Most of the tables do not have the bottom line, and the captions are to the bottom of the tables (I think table captions should be above the table itself, for ICLR papers). The spaces are not well organized, some tables and figures and texts are too close. All the experiment figures look the same: x, y axles with colored points. The presentation quality is far from an ICLR paper.
Questions for authors
What is the use case of the proposed method? Who will use it, in which scenatio?
Why the users do not know which task(s) is the LLM fine-tuned on?
If we can use hidden states to cluster the models, why we are not using the model parameters (or maybe some layers' parameters)?
How does the method scale when we have more tasks (e.g., 100 datasets)? How does the method scale when the fine-tuning hyperparameters are different, or even the fine-tuning method is different? For example, LoRA fine-tuning, quantized model?