- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Energy Transfer Detection (ETD), a method for detecting pretraining data from LLMs. The proposed method integrates prediction loss and predictive entropy by selecting the first-occurrence tokens. The authors evaluate the effectiveness of the proposed method across various datasets and models.
Critical strengths
The authors investigate predictive entropy as an additional signal for calibrating loss-based membership signals, which can benefit pretraining data detection.
Critical weaknesses
-
The theoretical motivation lacks justification. In this paper, the theoretical analysis mainly establishes variance reduction under specific assumptions. However, lower variance does not necessarily indicate better training data detection performance. The connection between the theoretical analysis and the proposed method remains unclear.
-
The experimental evaluation is not solid. In the main text, the experimental evaluation is primarily based on WikiMIA and StackMIAsub. Existing works show that benchmark post-hoc b construction can introduce distribution shifts, which can artificially inflate detection performance. The authors should further conduct experiments on more reliable benchmarks. For instance, the dataset used in "LLM Dataset Inference: Did You Train on My Dataset?" could be a more reliable benchmark for evaluation.
Questions for authors
-
What unique insights or practical benefits does the free-energy interpretation provide for the detection score?
-
In Theorem 2, how does variance reduction ensure improvements in AUC?
-
In the main experiments, how is the hyperparameter determined?
5 comments
Thank you for your detailed rebuttal. However, I am still confused about some points.
-
A higher AUROC lower bound does not guarantee improvement in the actual AUC. For example, based on results in Table 7, the Cantelli-based lower bound increases from 0.182 to 0.241, both well below the AUC of 0.5 (random guessing). Therefore, I do not think this bound establishes a meaningful connection between variance reduction and a higher actual AUC.
-
For W2, could you provide detailed results for each subset in your revised manuscript?
-
I still do not understand how the free-energy interpretation helps understand training data detection. Is the connection between free energy and the proposed detection score a similarity in mathematical form?
-
Additionally, could the authors specify how the reported 95% bootstrap confidence intervals were computed?
We sincerely thank you for carefully considering our previous responses and providing further constructive feedback. We have further strengthened the theoretical analysis, added detailed experimental results, and clarified the statistical procedures in the revised manuscript. Below are our point-by-point responses.
Re: Q1
Thank you for this important clarification. We agree that a higher AUROC lower bound does not necessarily guarantee an improvement in actual AUROC. Our previous analysis used Cantelli's inequality to establish a distribution-free guarantee without assuming any specific score-gap distribution, which explains why the resulting lower bound is conservative.
To establish a more direct connection with actual AUROC, we further derive Proposition 4 (AUROC Improvement under Approximate Gaussian Score Gaps) (Line 2164). Under a quantified approximate Gaussian assumption, Proposition 4 establishes the following bound:
\geq \Phi(\gamma_{\mathcal C})-\Phi(\gamma_{\mathcal L}) -(\epsilon_{\mathcal C}+\epsilon_{\mathcal L})$$ where $\Delta AUROC = AUROC_{ETD} - AUROC_{Loss}$ and $\gamma_S=\mu_t/\sigma_{S,t}$. Specifically, when the member–non-member score gaps approximately follow Gaussian distributions and the approximation errors are sufficiently small, preserving a positive mean gap while reducing its variance guarantees a strictly higher actual AUROC for ETD, rather than merely a higher lower bound. Based on the results in Table 7, **using the Gaussian approximation associated with Proposition 4 yields a predicted increase in average AUROC from 65.46% (loss-only) to 67.52% (ETD), an improvement of 2.06 percentage points** across 16 settings. The predicted ETD AUROC also closely matches the observed average of 67.54%. **These results empirically support the connection between improved standardized separation and higher actual AUROC.** We thank you again for this insightful comment, which has helped us strengthen the theoretical analysis. > **Re: Q2** Thank you for your suggestion. In our previous revision, we focused on the aggregated results across the 20 subsets to provide a concise comparison of overall detection performance. We agree that subset-level results are also important for assessing performance variations across different data domains. Therefore, we have now included detailed AUROC and TPR@5%FPR results for all 20 subsets in Appendix N (Figures 15–16, Lines 2538) of the revised manuscript. > **Re: Q3** Thank you for your follow-up. **The free-energy connection goes beyond algebraic similarity.** We previously explored this connection in depth but did not elaborate on it due to space constraints. We have now included the detailed analysis in the updated manuscript. As established in Proposition 5 (Pretraining as Non-equilibrium Free-Energy Minimization, Line 2220), defining the model-induced energy $\mathcal E_\theta(x)=-T\log P_\theta(x)$ yields the following non-equilibrium free-energy functional: $$\mathcal F_{\mathrm{neq}} =\mathbb E_{P_{\mathrm{data}}}[\mathcal E_\theta(x)] -T H(P_{\mathrm{data}}) =T D_{\mathrm{KL}}(P_{\mathrm{data}}\Vert P_\theta).$$ Consequently, **minimizing the maximum-likelihood pretraining objective is equivalent to minimizing this free-energy functional**: $$\arg\min_{\theta}\mathcal F_{\mathrm{neq}}(P_{\mathrm{data}},\theta)= \arg\min_{\theta}\mathcal L_{\mathrm{CE}}(\theta).$$ This establishes **a rigorous mathematical connection between maximum-likelihood pretraining and free-energy minimization**. During pretraining, the model learns to assign higher probabilities and thus lower prediction losses to training samples. From the free-energy perspective, this reduces their energy-like loss contributions. **This provides a theoretical motivation for why members tend to exhibit lower residual free-energy scores than non-members.** The complete derivation is provided in Appendix M. > **Re: Q4** We used 10,000 class-stratified bootstrap resamples within each subset, shared across methods and models. Metrics and averages were recomputed for each resample. The reported intervals are the point estimate ± 1.96 times the bootstrap standard error. We sincerely appreciate your continued engagement and constructive suggestions. We hope that these additional theoretical results, detailed experiments, and methodological clarifications can help address your remaining concerns. If anything remains unclear, we'd be happy to discuss it further.
We sincerely appreciate your recognition of predictive entropy as a useful calibration signal and thank you for your constructive questions. Below is our point-by-point response:
A1: Thank you for this insightful comment. We agree that lower variance alone does not guarantee better detection. However, when the positive expected member–non-member gap is preserved, variance reduction yields more stable separation and a higher distribution-free AUROC lower bound, as demonstrated below.
Following Eq.(4) (Line 181), the member–non-member score gap is ΔCt=C(non-members)−C(members), where ΔLt denotes the corresponding gap under conventional loss-only scoring. Theorems 1 & 2 show that entropy correction can preserve a positive expected gap while reducing its variance, thereby improving standardized separation (Eq. (10)):
{\sqrt{\operatorname{Var}(\Delta\mathcal C_t)}}> \frac{\mathbb E[\Delta L_t]} {\sqrt{\operatorname{Var}(\Delta L_t)}}.$$ Furthermore, using Cantelli's inequality, **we establish a distribution-free lower bound on AUROC:** $$\mathrm{AUROC}_S \geq P(\Delta S_t>0) \geq \frac{\mathbb E[\Delta S_t]^2} {\operatorname{Var}(\Delta S_t)+\mathbb E[\Delta S_t]^2}= \frac{\gamma_S^2}{1+\gamma_S^2},$$ where $\Delta S_t$ denotes the score gap between a non-member and a member, and $\gamma_S=\mathbb E[\Delta S_t]/\sqrt{\operatorname{Var}(\Delta S_t)}>0$ is the standardized separation. Since $\gamma^2/(1+\gamma^2)$ is strictly increasing for $\gamma>0$, Eq.(10) implies that **ETD has a strictly higher AUROC lower bound than loss-only scoring.** As shown in Table 4, entropy correction reduces the member–non-member gap variance by 53.14% / 52.34%, while improving standardized separation by 19.18% / 16.20% (Ori./Para.), **empirically supporting this mechanism.** We provide the complete proof in Appendix L (How Lower Variance Relates to Better Detection Performance). Thank you again for this insightful and important feedback! > **Q2:** The authors should further conduct experiments on more reliable benchmarks. For instance, the dataset used in "LLM Dataset Inference: Did You Train on My Dataset?" could be a more reliable benchmark for evaluation. (Weakness 2) **A2:** Thank you for this valuable suggestion. We evaluate ETD on the distribution-matched Pile-derived dataset described in [1], covering 20 subsets and four Pythia model scales: [Table A - Average AUROC & TPR@5%FPR] | Method | AUROC ↑ | TPR@5%FPR ↑ | |---|---:|---:| | PPL | 49.83 ± 0.51 | 5.70 ± 0.43 | | Ref | 51.10 ± 0.50 | 5.91 ± 0.39 | | Lowercase | 50.84 ± 0.49 | 6.18 ± 0.39 | | Zlib | 50.40 ± 0.50 | 5.60 ± 0.38 | | Min-K% | 50.93 ± 0.50 | 5.81 ± 0.45 | | Min-K%++ | 53.44 ± 0.46 | 5.83 ± 0.38 | | AECA | 50.89 ± 0.50 | 5.72 ± 0.44 | | DC-PDD | 50.22 ± 0.49 | 5.33 ± 0.34 | | ETD | **55.03 ± 0.46*** | **7.40 ± 0.40*** | As shown, **ETD outperforms the strongest baselines by 1.59% and 1.22% in AUROC and TPR@5%FPR, with statistically significant improvements.** Full results are provided in Appendix M of the updated manuscript. Thank you for your valuable feedback! > **Q3:** What unique insights or practical benefits does the free-energy interpretation provide for the detection score? (Question 1) **A3:** Thank you for the question. The free-energy perspective provides a macroscopic interpretation of the relationship between pretraining exposure and membership detection. As discussed in Section 4.4 (Lines 261–267), in thermodynamics, when a system is brought into contact with a reservoir, it relaxes toward equilibrium by dissipating free energy. Analogously, **pretraining members have previously interacted with the model and may involve less residual free-energy transfer during inference.** Thus, lower scores suggest stronger membership evidence, consistent with Figure 1. > **Q4:** In the main experiments, how is the hyperparameter determined? (Question 3) **A4:** Thank you for pointing this out. Detailed hyperparameter settings are provided in **Appendix D.2 (Implementation Details, Line 1013)**. We will highlight this in the main text. For $\lambda$, Theorem 2 derives the theoretically optimal coefficient $\lambda_t^*=\operatorname{Cov}(\Delta L_t,\Delta H_t)/\operatorname{Var}(\Delta H_t)$. For simplicity, we fix $\lambda=0.5$ across all datasets and target models in the main experiments, without specific tuning. We hope these clarifications and additional analyses can address your concerns. Thank you again for your constructive feedback. Please feel free to reach out if you have any further concerns! Reference [1] Maini et al. LLM Dataset Inference: Did You Train on My Dataset? NeurIPS 2024.