- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies distribution-matching distillation for continuous diffusion language models and proposes two methods: Simplex-DMD and Reinforce-DMD. Simplex-DMD treats token probability vectors as continuous representations and optimizes the objective using pathwise gradients. Reinforce-DMD instead samples discrete tokens, uses REINFORCE, and employs a discriminator to estimate the density ratio between the student and data distributions. The authors further extend the distribution-matching objective to multi-step generation and provide theoretical analyses. Experiments are conducted on OpenWebText using a pretrained LangFlow model as the teacher. At matched unigram entropy, Simplex-DMD reduces generative perplexity from 90.2 for ReDi to 45.6 with four network function evaluations (NFEs). Reinforce-DMD reduces it from 18.6 for D-MMD to 14.9 with 256 NFEs. Both methods achieve competitive results at different sampling budgets. The main contribution is the application of distribution-matching distillation to continuous diffusion language models, together with two approaches based on different gradient estimators. However, the current experiments do not sufficiently establish the robustness of the reported improvements or whether improvements in the evaluation metrics translate into better text quality.
Critical strengths
- The paper develops a unified distribution-matching framework and derives gradient estimators for both continuous relaxation and discrete sampling. Theorem 1 and the derivations in Appendix D help clarify the relationships among the objective, auxiliary estimators, and multi-step generation.
- Table 2 shows that Simplex-DMD achieves the lowest generative perplexity at matched entropy for 2, 4, 8, and 16 NFEs, while Reinforce-DMD performs best at 128 and 256 NFEs. In particular, at four NFEs, Simplex-DMD achieves a generative perplexity of 45.6, compared with 90.2 for ReDi, a reduction of approximately 49%. This result suggests that the proposed approach has potential for reducing the sampling cost of continuous diffusion language models.
Critical weaknesses
- The experiments are conducted only on OpenWebText and use a single random seed (Appendix E.1). Consequently, the current results are insufficient to establish the stability of the reported improvements or their generalization across datasets. The authors should provide repeated experiments or evaluations on additional datasets to further support the main claims.
- The paper primarily relies on GPT-2 Large generative perplexity and unigram entropy to evaluate generation quality and diversity. The samples in Appendix E.6 exhibit varying degrees of incoherence for both the proposed methods and the baselines. These examples alone do not justify concluding that the proposed methods generate worse text. However, the current metrics are also insufficient to establish that the actual quality of generated text has improved. The authors should provide more direct evaluations of text quality, such as blinded human evaluation or an independent assessment of textual coherence, and report the corresponding results.
Questions for authors
- The experiments are conducted only on OpenWebText and use a single random seed. Could the authors provide repeated experiments or evaluations on additional text datasets to further establish the stability of the reported improvements or their generalization across datasets?
- The samples in Appendix E.6 exhibit varying degrees of incoherence for both the proposed methods and the baselines. Have the authors conducted human evaluations or other direct assessments of text quality? If not, could they provide such evaluations to determine whether the improvements in generative perplexity and unigram entropy correspond to improvements in actual text quality?
3 comments
Rebuttal 2
2. Do the gains correspond to better text? (W2, Q2)
We did not run a human study. We followed your suggestion of an alternative assessment, with two measures that do not rely on GPT-2 perplexity.
MAUVE. It compares generated text with human text at the level of the whole corpus. We used 5,000 generated and 5,000 reference sequences featurized with GPT-2 large and quantized into 500 k-means clusters using 5 random seeds, and we report the results in Table D.
MAUVE supports the gains of S-DMD at 2, 4, and 8 NFE; at 4 NFE, for example, it reaches 71.9 versus 43.5 for ReDi. The rankings change at 16 NFE, where IDLM takes first place, and at 128–256 NFE, where D-MMD leads. Our methods rank second at 16 NFE (S-DMD) and third at 128–256 NFE (R-DMD), despite achieving lower Gen PPL.
Sudoku. The solve rate is exact: a generated grid is either correct or not (Table B in Rebuttal 1).
Your review raised two concerns: the robustness of the improvements and whether they correspond to better text. The additional datasets and sampling-seed study strengthen the evidence for robustness. Beyond Gen PPL, MAUVE corroborates the few-step gains of S-DMD, and Sudoku demonstrates improvements in accuracy on a task with an exact correctness criterion at small sampling budgets.
We hope these additions address your concerns and would be grateful if you would consider updating your assessment.
Table D — OWT: Gen PPL / MAUVE () at matched unigram entropy . A small grey value in parentheses is the entropy reached when a method cannot reach the target. Gen PPL: bold = lowest. MAUVE: bold = highest, italic = second highest (among matched values).
| Method | 2 NFE | 4 NFE | 8 NFE | 16 NFE | 128 NFE | 256 NFE |
|---|---|---|---|---|---|---|
| 5.65 | 5.44 | 5.38 | 5.20 | 5.27 | 5.00 | |
| LangFlow (teacher) | 1077 / 0.7 | 317 / 2.2 | 147 / 11.0 | 58.6 / 38.3 | 39.1 / 85.9 | 21.3 / 80.0 |
| MDLM | 1909 / 0.5 | 678 / 0.6 | 267 / 2.2 | 72.8 / 16.5 | 44.3 / 75.8 | 23.8 / 66.7 |
| FMLM | 142 / 1.5 | 128 / 4.8 | 94.7 / 12.7 | 55.2 / 9.5 | 22.6 / 10.0 | 13.5 / 3.1 |
| D-MMD | 1752 / 0.5 | 431 / 1.1 | 135 / 11.2 | 41.7 / 47.7 | 27.3 / 89.8 | 18.6 / 84.5 |
| IDLM | 955 / 0.6 | 145 / 7.6 | 48.4 / 62.7 | 24.3 / 85.0 | 11.6 / 65.5 | 9.14 / 51.9 |
| SDTT | 1496 / 0.5 | 372 / 0.9 | 103 / 11.8 | 39.4 / 34.6 | 32.6 / 77.9 | 20.1 / 60.0 |
| ReDi | 261 / 2.9 | 90.2 / 43.5 | 49.4 / 66.5 | 32.4 / 64.2 | 30.7 / 59.5 | 25.0 / 35.7 |
| S-DMD (ours) | 178 / 16.6 | 45.6 / 71.9 | 26.7 / 77.4 | 17.0 / 70.7 | 3.60 / 9.3 | 2.51 / 3.6 |
| R-DMD (ours) | 1112 / 0.7 | 556 / 1.0 | 347 / 1.3 | 68.8 / 24.2 | 26.9 / 85.7 | 14.9 / 75.0 |
Thanks for the additional results. Most of my concerns have been addressed. Therefore, I raise my overall score to 3.
Rebuttal 1
Thank you for the review. You raised two concerns; here is what we did for each. This first comment covers datasets and seeds; the second covers text quality beyond Gen PPL.
1. One dataset and one seed (W1, Q1)
More datasets. We added LM1B and Sudoku. The few-step gains observed on OWT are also present on both.
LM1B (Table A). We use the released LangFlow LM1B teacher and the same protocol as Table 2 of the paper. The takeaway is similar to OWT; e.g., at 4 NFE and matched entropy, S-DMD reaches a Gen PPL of 111, against 361 for its teacher and 328 for D-MMD.
Sudoku (Table B). We measure the solve rate over 100,000 test puzzles. At 1 NFE, S-DMD solves 78.6% of the puzzles, against 69.9% for its teacher. From 2 NFE on, our two students have the highest solve rates of all methods (S-DMD at 2 NFE, R-DMD at 4 and 8 NFE).
Seeds. On OWT, each point in our tables is computed from 32 generated sequences of 1,024 tokens. This number was missing from the paper; we will add it. To measure the noise of this protocol, we repeated it with 5 sampling seeds for 5 different methods from 1 to 1,024 NFE (Table C). Across seeds, Gen PPL varies by 3–5% and unigram entropy by about 1% or less. The gaps we report are much larger: 49% at 4 NFE and 20% at 256 NFE. This low variance is consistent with the common practice in this literature of reporting a single evaluation run, and a fortiori a single training run.
Table A — LM1B: Gen PPL (lower is better) at matched unigram entropy. A small grey value in parentheses is the entropy reached when a method cannot reach the target.
Table B — Sudoku: solve rate (%) on 100k test puzzles.
Table C — OWT: sampling-seed variability of Gen PPL at temperature 1.0. Coefficient of variation (std / mean over 5 seeds, 32 sequences each), averaged over 2 to 1,024 NFE.