- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Gradient Deviation Scores (GDS), a method designed to detect pretraining data used by large language models. Specifically, GDS uses a lightweight classifier to perform binary membership inference. The authors evaluate the effectiveness of the proposed method on various datasets and models.
Critical strengths
- The authors explore potential membership signals derived from sample gradients on target models, which is a valuable attempt at training data detection.
Critical weaknesses
-
The motivation for the proposed method lacks sufficient justification First, the empirical motivation is established by analyzing parameter updates during LoRA fine-tuning. However, the detection method uses parameter gradients from randomly initialized LoRA adapters. The practical equivalence between these two distinct mechanisms remains unclear. Second, a sound motivation should demonstrate that members and non-members exhibit distinguishable behaviors on the target model. The authors, however, solely analyze member dynamics during fine-tuning. Finally, pre-training data detection is challenging due to the use of single-epoch training. In contrast, the observations presented in the motivational study stem from multi-epoch fine-tuning (e.g., the progressive stabilization of update locations). It remains unclear how these findings apply to pretraining settings.
-
The manuscript requires substantial improvement in presentation The authors should use standard mathematical notation in the manuscript. For example, “Row_Mean_Std” should be expressed using mathematical notation in the equations. Citation formatting is noticeably inconsistent. Some citations include publication years, while others do not. The authors should carefully revise the captions to ensure they are clear and informative. Overall, the manuscript contains numerous formatting issues and typesetting artifacts, which significantly affect its readability.
-
The experiments need further strengthening. The distribution shift in the evaluation set may affect the evaluation results in training data detection. The dataset used in “LLM Dataset Inference: Did You Train on My Dataset?” could be a more suitable choice for evaluation.
Suggestions: Please use AI tools thoughtfully and critically. An acceptable paper still requires substantial human effort in its structure, writing, and formatting.
Questions for authors
Questions:
-
In Figure 1, what metric does the y-axis represent? Why are the x-axis and y-axis labeled identically in the left panel?
-
As stated in the paper, “For each training sample, we perform inference and backpropagation to obtain LoRA gradients.” For pretrained models, where are the LoRA modules?
-
The authors analyze parameter updates via LoRA fine-tuning in the motivation experiments, while they use randomly initialized LoRA for the actual detection. Are these two approaches actually equivalent?
-
Why is the largest change in average update magnitude observed at exposure 0? Exposure 0 indicates that fine-tuning does not occur.
-
Why are some of the best results highlighted in bold, while others are not? For example, the experimental results in Table 10.
-
The caption notes that “Bold denotes an obvious difference between exposure 0 and 1.” However, the filled markers in Figures 2c and 2d appear to indicate the minimum values.
-
Please check the citation formatting. Some citations include publication years, while others do not. This makes the citations appear inconsistent throughout the manuscript.