- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a data extraction framework named D-Miner targeting Masked Diffusion Language Models (DLMs). The framework extracts post-training alignment data by inserting a lightweight hidden-layer corrector and guiding the generation process using a self-generated MScore signal.
Critical strengths
Novel Problem and Approach: Exploring the data extraction risks of DLMs starting from a pure mask state is an interesting and relatively new problem. The idea of guiding generation through lightweight feature shifts in the hidden layers is also somewhat inspiring.
Critical weaknesses
- I am not an expert in the security domain, but I have concerns regarding the realistic plausibility of the paper's settings. The paper assumes that an attacker can arbitrarily tamper with the activation values of intermediate layers (e.g., layer 16) during decoding. However, in a black-box API scenario, the attacker cannot access the hidden states, rendering this method completely ineffective. In an open-source white-box scenario, since the attacker already possesses the complete model weights, they can typically employ more efficient white-box attack methods (such as gradient analysis, weight inversion, or fine-tuning) to extract information. The authors need to provide a more compelling argument for why this specific threat scenario is indispensable in the real world.
- This paper uses "semantic matching" to evaluate the success rate of extraction. If the model generates high-quality responses that are semantically consistent with the training set but differ in exact wording, this actually proves that the model is well-aligned and generalizes well. Equating this with "training data leakage" appears to be a conflation of concepts.
Questions for authors
Same as Critical Weaknesses
3 comments
Response to Reviewer 8rXZ (2/2)
To W2: Generalization vs. Data Extraction
“If the model generates high-quality responses that are semantically consistent with the training set but differ in exact wording, this actually proves that the model is well-aligned and generalizes well.”
We believe that generalization in capability and leakage of private data do not conflict: a model can generalize well and still expose its post-training data. To verify this, we conduct an experiment on ESPO-MATH that compares outputs reflecting generalization with the outputs of D-Miner.
Setting. The high-quality outputs that reflect generalization are the 197 correct responses of ESPO-MATH to the MATH-500 test problems (held out from its training data). These responses and the D-Miner outputs are matched against the post-training data of ESPO-MATH with the same encoder and threshold as Table 2 in the paper, and each response is matched both with and without its problem to separate the contribution of the given problem. The metric is the fraction of outputs whose nearest post-training record has similarity ≥ 0.90, so a higher fraction means that more outputs are close to the post-training data.
If semantic matching only rewarded high-quality responses that generalize, as the reviewer suggests, the correct responses would be matched to the post-training data at least as often as the outputs of D-Miner. Table 2 shows the opposite:
- The correct responses are the generalization case that the reviewer describes. They are semantically consistent with the reference solutions but differ in wording: none of them is identical to its reference solution, while their mean embedding similarity to it is 0.89. The model produces them only as answers to the problems we provide.
- Generalization yields only a limited match rate. Even with the problem included, 23.9% of the correct responses are matched, and the rate drops to 18.3% when only the response is matched. Part of their matches thus comes from the problem we provide rather than from the generated solution.
- D-Miner is matched more often, without any input. D-Miner receives no problem, yet 31.8% of its outputs are matched. Generalization alone therefore does not explain the matches of D-Miner.
Overall, capability and leakage coexist in the same model: generalization appears only when a problem is given, whereas D-Miner exposes the post-training data without any input.
Table 2. Fraction of outputs matched to the post-training data of ESPO-MATH (similarity ≥ 0.90).
| Output | Sim ≥ 0.90 |
|---|---|
| Correct responses on MATH-500 (problem + response) | 23.9% |
| Correct responses on MATH-500 (response only) | 18.3% |
| D-Miner | 31.8% |
Dear Reviewer 8rXZ,
We are writing to ask whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining issues.
Response to Reviewer 8rXZ (1/2)
We appreciate the reviewer's recognition of the problem and of the hidden-state guidance idea.
To W1: Threat-Model Practicality
We would like to clarify that our access assumption reflects a common deployment setting. Leading models such as DeepSeek and GLM release their weights for local deployment, while their post-training data remain unavailable. The DLMs we study (Dream, LLaDA, ESPO) are released in the same way. The attacker therefore holds the released weights but not the post-training data, nor any artifact of the training process such as gradients. Black-box APIs are outside our scope.
The white-box methods mentioned by the reviewer assume access beyond the released weights:
D-Miner requires none of these: it uses only the released weights and starts from a fully masked input.
Although these assumptions do not hold in our setting, we transfer the gradient-based and fine-tuning-based methods to the released weights to test whether they recover more training records than D-Miner. Both start from the same candidates selected by MScore as D-Miner, so that the comparison differs only in how the candidates are used, and each generates 1,024 outputs per seed over three seeds:
As Table 1 shows, D-Miner recovers more training records than both transferred methods on all four targets, even though they are given the same candidates and may optimize the input or update the model weights. Gradient-based optimization stays below Simple on all four targets, and fine-tuning exceeds Simple only on the two LLaDA targets. Thus, the white-box methods that the reviewer mentions either require information that released models do not provide or recover fewer training records than D-Miner.
Table 1. UMS of the transferred white-box methods (mean over three seeds; Simple and D-Miner from Table 2 in the paper).
[1] Zhu et al. Deep Leakage from Gradients. NeurIPS 2019.
[2] Balunović et al. LAMP: Extracting Text from Gradients with Language Model Priors. NeurIPS 2022.
[3] Petrov et al. DAGER: Exact Gradient Inversion for Large Language Models. NeurIPS 2024.
[4] Haim et al. Reconstructing Training Data from Trained Neural Networks. NeurIPS 2022.
[5] Chen et al. The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks. CCS 2024.
[6] Zhang et al. Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation. ACL 2023.