acceptodds
Under review as a conference paper at ICLR 2027

Distribution Matching Distillation for Continuous Diffusion Language Models

Abstract

Continuous diffusion language models are becoming competitive with their discrete counterparts, but generation still requires many sequential network evaluations. Trajectory-based distillation methods have been developed to reduce this cost, while distributional distillation remains underexplored in this setting, particularly in exploiting the discrete structure of language and enabling multi-step generation. We develop a unified formulation of existing distributional distillation approaches that clarifies how the student’s probabilistic parameterization determines the available gradient estimators. Building on this formulation, we introduce two methods that share the same student architecture and reverse-KL objective: Simplex-DMD uses a continuous token relaxation and pathwise gradients, while Reinforce-DMD uses categorical token sampling and REINFORCE with a learned density ratio. We extend both methods to multi-step generation and investigate the training and sampling choices associated with each parameterization. On OpenWebText, using a sequence length of 1,024 tokens, Simplex-DMD and Reinforce-DMD improve the generative perplexity–entropy frontier over competing methods at complementary sampling budgets. In particular, at comparable entropy, Simplex-DMD reduces generative perplexity by 49% at 4 network evaluations, while Reinforce-DMD achieves a 20% reduction at 256 evaluations, each relative to the best evaluated baseline at the same budget.

Primary area:
generative models
Keywords:
diffusion language models, distribution matching distillation, continuous diffusion, discrete data, few-step generation, continuous relaxation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_v5keOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies distribution-matching distillation for continuous diffusion language models and proposes two methods: Simplex-DMD and Reinforce-DMD. Simplex-DMD treats token probability vectors as continuous representations and optimizes the objective using pathwise gradients. Reinforce-DMD instead samples discrete tokens, uses REINFORCE, and employs a discriminator to estimate the density ratio between the student and data distributions. The authors further extend the distribution-matching objective to multi-step generation and provide theoretical analyses. Experiments are conducted on OpenWebText using a pretrained LangFlow model as the teacher. At matched unigram entropy, Simplex-DMD reduces generative perplexity from 90.2 for ReDi to 45.6 with four network function evaluations (NFEs). Reinforce-DMD reduces it from 18.6 for D-MMD to 14.9 with 256 NFEs. Both methods achieve competitive results at different sampling budgets. The main contribution is the application of distribution-matching distillation to continuous diffusion language models, together with two approaches based on different gradient estimators. However, the current experiments do not sufficiently establish the robustness of the reported improvements or whether improvements in the evaluation metrics translate into better text quality.

Critical strengths
  1. The paper develops a unified distribution-matching framework and derives gradient estimators for both continuous relaxation and discrete sampling. Theorem 1 and the derivations in Appendix D help clarify the relationships among the objective, auxiliary estimators, and multi-step generation.
  2. Table 2 shows that Simplex-DMD achieves the lowest generative perplexity at matched entropy for 2, 4, 8, and 16 NFEs, while Reinforce-DMD performs best at 128 and 256 NFEs. In particular, at four NFEs, Simplex-DMD achieves a generative perplexity of 45.6, compared with 90.2 for ReDi, a reduction of approximately 49%. This result suggests that the proposed approach has potential for reducing the sampling cost of continuous diffusion language models.
Critical weaknesses
  1. The experiments are conducted only on OpenWebText and use a single random seed (Appendix E.1). Consequently, the current results are insufficient to establish the stability of the reported improvements or their generalization across datasets. The authors should provide repeated experiments or evaluations on additional datasets to further support the main claims.
  2. The paper primarily relies on GPT-2 Large generative perplexity and unigram entropy to evaluate generation quality and diversity. The samples in Appendix E.6 exhibit varying degrees of incoherence for both the proposed methods and the baselines. These examples alone do not justify concluding that the proposed methods generate worse text. However, the current metrics are also insufficient to establish that the actual quality of generated text has improved. The authors should provide more direct evaluations of text quality, such as blinded human evaluation or an independent assessment of textual coherence, and report the corresponding results.
Questions for authors
  1. The experiments are conducted only on OpenWebText and use a single random seed. Could the authors provide repeated experiments or evaluations on additional text datasets to further establish the stability of the reported improvements or their generalization across datasets?
  2. The samples in Appendix E.6 exhibit varying degrees of incoherence for both the proposed methods and the baselines. Have the authors conducted human evaluations or other direct assessments of text quality? If not, could they provide such evaluations to determine whether the improvements in generative perplexity and unigram entropy correspond to improvements in actual text quality?
3 comments
AuthorsOpenReview ↗

Rebuttal 1

Thank you for the review. You raised two concerns; here is what we did for each. This first comment covers datasets and seeds; the second covers text quality beyond Gen PPL.

1. One dataset and one seed (W1, Q1)

More datasets. We added LM1B and Sudoku. The few-step gains observed on OWT are also present on both.

LM1B (Table A). We use the released LangFlow LM1B teacher and the same protocol as Table 2 of the paper. The takeaway is similar to OWT; e.g., at 4 NFE and matched entropy, S-DMD reaches a Gen PPL of 111, against 361 for its teacher and 328 for D-MMD.

Sudoku (Table B). We measure the solve rate over 100,000 test puzzles. At 1 NFE, S-DMD solves 78.6% of the puzzles, against 69.9% for its teacher. From 2 NFE on, our two students have the highest solve rates of all methods (S-DMD at 2 NFE, R-DMD at 4 and 8 NFE).

Seeds. On OWT, each point in our tables is computed from 32 generated sequences of 1,024 tokens. This number was missing from the paper; we will add it. To measure the noise of this protocol, we repeated it with 5 sampling seeds for 5 different methods from 1 to 1,024 NFE (Table C). Across seeds, Gen PPL varies by 3–5% and unigram entropy by about 1% or less. The gaps we report are much larger: 49% at 4 NFE and 20% at 256 NFE. This low variance is consistent with the common practice in this literature of reporting a single evaluation run, and a fortiori a single training run.

Table A — LM1B: Gen PPL (lower is better) at matched unigram entropy. A small grey value in parentheses is the entropy reached when a method cannot reach the target.

Method2 NFE4 NFE8 NFE32 NFE64 NFE128 NFE
Target entropy4.384.344.314.304.304.31
LangFlow (teacher)759361192100.093.391.0
MDLM987449222125109109
FMLM110 (4.20)\color{gray}{\scriptstyle (4.20)}95.5 (4.20)\color{gray}{\scriptstyle (4.20)}83.6 (4.21)\color{gray}{\scriptstyle (4.21)}73.3 (4.24)\color{gray}{\scriptstyle (4.24)}69.3 (4.24)\color{gray}{\scriptstyle (4.24)}57.9 (4.20)\color{gray}{\scriptstyle (4.20)}
D-MMD87332817599.088.285.4
S-DMD (ours)24711173.860.6 (4.29)\color{gray}{\scriptstyle (4.29)}56.5 (4.28)\color{gray}{\scriptstyle (4.28)}56.1 (4.28)\color{gray}{\scriptstyle (4.28)}
R-DMD (ours)59024812379.476.785.8

Table B — Sudoku: solve rate (%) on 100k test puzzles.

TeacherMethod1 NFE2 NFE4 NFE8 NFE
MDLMMDLM (teacher)84.0696.9498.4799.12
MDLMD-MMD89.3998.2198.9799.44
MDLMIDLM89.5098.2298.9799.44
LangFlowLangFlow (teacher)69.8997.3099.6199.89
LangFlowS-DMD (ours)78.5598.5399.6799.79
LangFlowR-DMD (ours)72.1897.6099.6899.90

Table C — OWT: sampling-seed variability of Gen PPL at temperature 1.0. Coefficient of variation (std / mean over 5 seeds, 32 sequences each), averaged over 2 to 1,024 NFE.

LangFlowMDLMUDLMS-DMD (ours)R-DMD (ours)
CV (%)4.33.74.14.74.9
AuthorsOpenReview ↗

Rebuttal 2

2. Do the gains correspond to better text? (W2, Q2)

We did not run a human study. We followed your suggestion of an alternative assessment, with two measures that do not rely on GPT-2 perplexity.

MAUVE. It compares generated text with human text at the level of the whole corpus. We used 5,000 generated and 5,000 reference sequences featurized with GPT-2 large and quantized into 500 k-means clusters using 5 random seeds, and we report the results in Table D.

MAUVE supports the gains of S-DMD at 2, 4, and 8 NFE; at 4 NFE, for example, it reaches 71.9 versus 43.5 for ReDi. The rankings change at 16 NFE, where IDLM takes first place, and at 128–256 NFE, where D-MMD leads. Our methods rank second at 16 NFE (S-DMD) and third at 128–256 NFE (R-DMD), despite achieving lower Gen PPL.

Sudoku. The solve rate is exact: a generated grid is either correct or not (Table B in Rebuttal 1).

Your review raised two concerns: the robustness of the improvements and whether they correspond to better text. The additional datasets and sampling-seed study strengthen the evidence for robustness. Beyond Gen PPL, MAUVE corroborates the few-step gains of S-DMD, and Sudoku demonstrates improvements in accuracy on a task with an exact correctness criterion at small sampling budgets.

We hope these additions address your concerns and would be grateful if you would consider updating your assessment.

Table D — OWT: Gen PPL / MAUVE (×100\times 100) at matched unigram entropy HNFEH_{\mathrm{NFE}}. A small grey value in parentheses is the entropy reached when a method cannot reach the target. Gen PPL: bold = lowest. MAUVE: bold = highest, italic = second highest (among matched values).

Method2 NFE4 NFE8 NFE16 NFE128 NFE256 NFE
HNFEH_{\mathrm{NFE}}5.655.445.385.205.275.00
LangFlow (teacher)1077 / 0.7317 / 2.2147 / 11.058.6 / 38.339.1 / 85.921.3 / 80.0
MDLM1909 / 0.5678 / 0.6267 / 2.272.8 / 16.544.3 / 75.823.8 / 66.7
FMLM142 / 1.5 (5.32)\color{gray}{\scriptstyle (5.32)}128 / 4.8 (5.40)\color{gray}{\scriptstyle (5.40)}94.7 / 12.755.2 / 9.522.6 / 10.0 (4.87)\color{gray}{\scriptstyle (4.87)}13.5 / 3.1 (4.50)\color{gray}{\scriptstyle (4.50)}
D-MMD1752 / 0.5431 / 1.1135 / 11.241.7 / 47.727.3 / 89.818.6 / 84.5
IDLM955 / 0.6145 / 7.648.4 / 62.724.3 / 85.011.6 / 65.5 (4.75)\color{gray}{\scriptstyle (4.75)}9.14 / 51.9 (4.27)\color{gray}{\scriptstyle (4.27)}
SDTT1496 / 0.5372 / 0.9103 / 11.839.4 / 34.632.6 / 77.920.1 / 60.0
ReDi261 / 2.9 (5.46)\color{gray}{\scriptstyle (5.46)}90.2 / 43.549.4 / 66.532.4 / 64.2 (5.32)\color{gray}{\scriptstyle (5.32)}30.7 / 59.525.0 / 35.7 (5.19)\color{gray}{\scriptstyle (5.19)}
S-DMD (ours)178 / 16.645.6 / 71.926.7 / 77.417.0 / 70.73.60 / 9.3 (3.82)\color{gray}{\scriptstyle (3.82)}2.51 / 3.6 (3.31)\color{gray}{\scriptstyle (3.31)}
R-DMD (ours)1112 / 0.7556 / 1.0347 / 1.368.8 / 24.226.9 / 85.714.9 / 75.0

Thanks for the additional results. Most of my concerns have been addressed. Therefore, I raise my overall score to 3.

Reviewer_J49POpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors consider distribution matching distillation (DMD) in the embedding space of continuous diffusion language models (LangFlow) to train a student model that should work with fewer network evaluations (NFEs). The distribution of texts after mapping through the embedding layer (as in LangFlow) is represented as a mixture of delta measures in the embedding space, with each sequence represented by a product of deltas over its positions. In this setting, the authors consider two parameterizations for the student model: the first is discrete and samples a specific token at each position, independently conditional on the noisy input, while the second is a simplex relaxation and outputs a point on the vocabulary simplex at each position. For the continuous relaxation, they use standard DMD and prove that if all embeddings are distinct and have the same norm, then zero DMD loss implies recovery of the text distribution. For the discrete case, the authors use REINFORCE with leave-one-out baselines inspired by GRPO, since pathwise differentiation is not possible as in the continuous case. The authors validate their approach on the standard OWT setup and use the generative perplexity-entropy frontier to compare it with other methods.

Critical strengths

Strong Gen PPL–entropy frontier results for Simplex-DMD in few-step sampling. Compared with other DLMs and their distilled variants, Simplex-DMD shows a clear advantage on the reported frontier.

The first study of DMD distillation for continuous DLMs. To my knowledge, this work is the first study of DMD distillation for continuous DLMs. Furthermore, the authors provide conditions under which matching the distributions of noised embeddings with a simplex relaxation guarantees matching the original text distribution at zero loss. Also, while REINFORCE is a standard way to optimize through discrete sampling, it is still interesting to see how the authors make this parameterization work using a standard leave-one-out baseline, inspired by GRPO, together with KL regularization.

Critical weaknesses

Limited methodological novelty Both proposed methods are a slight modification of a standard DMD using standard components. Simplex-DMD uses the reverse-KL objective with its gradient via the difference of the score function and reparametrization technique for optimization. Reinforce-DMD uses a standard REINFORCE with a leave-one-out baseline and KL regularization from GRPO. The multi-step regime with (forward diffusion) is also standard and was studied in the original DMD works [3, Section 4.4], which explicitly alternates clean-sample prediction with fresh Gaussian noise injection. Although the present formulation has differences, including conditioning the auxiliary model on the starting time TT, which is omitted in the DMD2 implementation, the core method is standard DMD or REINFORCE with GRPO baselines for DMD with discrete distributions as the objective.

Limited experimental validation The authors provide results only for the OWT setup with a single LangFlow teacher and use only gen ppl-entropy frontiers. However, gen ppl-entropy frontiers can be hacked by simple, "dummy" generators. For instance, [4] shows that incoherent periodic generators can substantially outperform trained models at nearly identical unigram entropy. Thus, controlling unigram entropy does not guarantee text quality or preservation of diversity. Furthermore, the provided Simplex-DMD samples contain boilerplate and repeated expressions, which can benefit Gen PPL because they are easy to predict, while the semantic coherence of the generated text remains weak, such as “chemical attack attack” and “phone numbers, phone numbers."

The evaluation would be stronger with additional metrics that cannot be easily hacked and that evaluate diversity across samples (not only diversity of words in samples as unigram entropy), such as MAUVE and corpus-level n-gram diversity, computed over the whole corpus and not averaged over individual texts. An additional dataset or task with a clear generation quality metric, such as TinyGSM, would also help establish whether the proposed approach accelerates generation. Other setups, such as Sudoku, also offer verifiable and interpretable metrics and would be beneficial, although language-based tasks such as TinyGSM would be more directly relevant.

1] Luo et al. Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models. NeurIPS, 2023.

[2] Yin et al. One-step Diffusion with Distribution Matching Distillation. CVPR, 2024.

[3] Yin et al. Improved Distribution Matching Distillation for Fast Image Synthesis. NeurIPS, 2024.

[4] Franca and Tong. Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics. ICML 2026 Workshop on Structured Probabilistic Inference & Generative Modeling.

Questions for authors

In DMD2, fake scores are trained without additional conditioning on the starting time used by the student to produce the clean sample before renoising Also, DMD2 uses backward simulation for its generation and ablate that it is better then forward simulation used in this paper. Did you try to ablate this for your model for language setups?

3 comments
AuthorsOpenReview ↗

Rebuttal 1

Thank you for the detailed and precise review. We agree with several of your points. This first comment covers the evaluation; the second covers the additional datasets, your question, and novelty.

1. Evaluation beyond the Gen PPL–entropy frontier (W2)

We agree that the Gen PPL–entropy frontier alone is not sufficient, and that repeated phrases such as the ones you quote can lower Gen PPL without making the text better. As suggested by the reviewer, we add another measure to control for these failure modes, by computing the MAUVE score. We use 5,000 generated and 5,000 reference sequences featurized with GPT-2 large and quantized into 500 k-means clusters using 5 random seeds, and we report the results in Table A. We will include this table in the revised version of the paper.

Table A — OWT: Gen PPL / MAUVE (×100\times 100) at matched unigram entropy HNFEH_{\mathrm{NFE}}. A small grey value in parentheses is the entropy reached when a method cannot reach the target. Bold: lowest Gen PPL and highest MAUVE among matched values.

Method2 NFE4 NFE8 NFE16 NFE128 NFE256 NFE
HNFEH_{\mathrm{NFE}}5.655.445.385.205.275.00
LangFlow (teacher)1077 / 0.7317 / 2.2147 / 11.058.6 / 38.339.1 / 85.921.3 / 80.0
MDLM1909 / 0.5678 / 0.6267 / 2.272.8 / 16.544.3 / 75.823.8 / 66.7
FMLM142 / 1.5 (5.32)\color{gray}{\scriptstyle (5.32)}128 / 4.8 (5.40)\color{gray}{\scriptstyle (5.40)}94.7 / 12.755.2 / 9.522.6 / 10.0 (4.87)\color{gray}{\scriptstyle (4.87)}13.5 / 3.1 (4.50)\color{gray}{\scriptstyle (4.50)}
D-MMD1752 / 0.5431 / 1.1135 / 11.241.7 / 47.727.3 / 89.818.6 / 84.5
IDLM955 / 0.6145 / 7.648.4 / 62.724.3 / 85.011.6 / 65.5 (4.75)\color{gray}{\scriptstyle (4.75)}9.14 / 51.9 (4.27)\color{gray}{\scriptstyle (4.27)}
SDTT1496 / 0.5372 / 0.9103 / 11.839.4 / 34.632.6 / 77.920.1 / 60.0
ReDi261 / 2.9 (5.46)\color{gray}{\scriptstyle (5.46)}90.2 / 43.549.4 / 66.532.4 / 64.2 (5.32)\color{gray}{\scriptstyle (5.32)}30.7 / 59.525.0 / 35.7 (5.19)\color{gray}{\scriptstyle (5.19)}
S-DMD (ours)178 / 16.645.6 / 71.926.7 / 77.417.0 / 70.73.60 / 9.3 (3.82)\color{gray}{\scriptstyle (3.82)}2.51 / 3.6 (3.31)\color{gray}{\scriptstyle (3.31)}
R-DMD (ours)1112 / 0.7556 / 1.0347 / 1.368.8 / 24.226.9 / 85.714.9 / 75.0

2. Additional datasets (W2)

LM1B (Table B). We use the released LangFlow LM1B teacher and the same protocol as Table 2 of the paper. The takeaway is similar to OWT; e.g., at 4 NFE and matched entropy, S-DMD reaches a Gen PPL of 111, against 361 for its teacher and 328 for D-MMD.

Sudoku (Table C). We measure the solve rate over 100,000 test puzzles. At 1 NFE, S-DMD solves 78.6% of the puzzles, against 69.9% for its teacher; the discrete students D-MMD and IDLM are ahead at this budget (89%). From 2 NFE on, our two students have the highest solve rates of all methods (S-DMD at 2 NFE, R-DMD at 4 and 8 NFE).

We agree that TinyGSM would be a relevant language task. We could not run it during the discussion period, because no continuous-diffusion teacher is publicly available for this dataset, and training one was beyond our compute budget. We intend to run this experiment in the revised version of the paper.

Table B — LM1B: Gen PPL at matched unigram entropy. Same convention as Table A.

Method2 NFE4 NFE8 NFE32 NFE64 NFE128 NFE
Target entropy4.384.344.314.304.304.31
LangFlow (teacher)759361192100.093.391.0
MDLM987449222125109109
FMLM110 (4.20)\color{gray}{\scriptstyle (4.20)}95.5 (4.20)\color{gray}{\scriptstyle (4.20)}83.6 (4.21)\color{gray}{\scriptstyle (4.21)}73.3 (4.24)\color{gray}{\scriptstyle (4.24)}69.3 (4.24)\color{gray}{\scriptstyle (4.24)}57.9 (4.20)\color{gray}{\scriptstyle (4.20)}
D-MMD87332817599.088.285.4
S-DMD (ours)24711173.860.6 (4.29)\color{gray}{\scriptstyle (4.29)}56.5 (4.28)\color{gray}{\scriptstyle (4.28)}56.1 (4.28)\color{gray}{\scriptstyle (4.28)}
R-DMD (ours)59024812379.476.785.8

Table C — Sudoku: solve rate (%) on 100k test puzzles.

TeacherMethod1 NFE2 NFE4 NFE8 NFE
MDLMMDLM (teacher)84.0696.9498.4799.12
MDLMD-MMD89.3998.2198.9799.44
MDLMIDLM89.5098.2298.9799.44
LangFlowLangFlow (teacher)69.8997.3099.6199.89
LangFlowS-DMD (ours)78.5598.5399.6799.79
LangFlowR-DMD (ours)72.1897.6099.6899.90
AuthorsOpenReview ↗

Rebuttal 2

3. Conditioning on TT and backward simulation (Q1)

Conditioning on TT. We thank the reviewer for raising this interesting question. Yes, our auxiliary denoiser and our discriminator both receive TT, as in Eqs. (11) and (12). Without TT, the auxiliary model fits the average of the student marginals over starting times, pˉtη=ET∣t[ptη,T]\bar p_t^{\eta} = \mathbb{E}_{T \mid t}\big[p_t^{\eta, T}\big], and the student matches this quantity. By convexity of the KL divergence, this is a weaker objective than ours. We have not yet run the variant without TT due to time constraints, and we will add this ablation to the paper.

Backward simulation. We thank the reviewer for letting us elaborate on this technique. Our training aims to learn a transport from pTp_T to p0p_0 in embedding space. Backward simulation replaces pTp_T with p^T sg⁡(η)\widehat p_T^{\,\operatorname{sg}(\eta)}, where p^T η\widehat p_T^{\,\eta} denotes the distribution of the student's sampling state at time TT, and sg⁡\operatorname{sg} stops gradients through the rollout. We tested this variant on S-DMD and observed complete collapse, with unigram entropy close to zero. We therefore retained inputs obtained by noising real data.

The practical benefit of backward rollout depends on the student's sensitivity to input mismatch and on the stability of training. Integrating this technique is a possible but non-trivial extension that we leave for future work.

4. Novelty (W1)

As you note in your review, this is, to our knowledge, the first study of distribution matching distillation for continuous diffusion language models. We agree that its building blocks (reverse-KL matching, the leave-one-out baseline, multi-step sampling with fresh noise) belong to the standard toolbox. We will clarify this point, and will also make sure to cite and discuss DMD2 in Section 3.2 and in the related work section of our revised paper.

Our point is that the standard recipe does not transfer to text as it is, and our work extends beyond these base components in order to make a proper methodological contribution:

  • We identify when matching noised embeddings recovers the data distribution, with Theorem 1.
  • We develop two student parameterizations, continuous and categorical, and show how these require different gradient estimators and exhibit distinct behavior across sampling budgets. We also identify the importance of KL regularization for stable categorical training in our experiments.
  • We train on noised real data and show this is stable and performant, while the previous remark shows how the backward simulation used in DMD2 does not straightforwardly transfer to our setting, leading to collapse. This identifies a limitation of directly adopting the image-distillation recipe and distinctively motivates our approach.
  • We show that each model can be used with an arbitrary number of sampling steps with simple time conditioning, whereas DMD2 needs a fixed sampling schedule.

The contribution therefore combines a theoretical justification for matching continuous representations of discrete data, two concrete distillation methods, and an empirical study of the choices that determine their effectiveness and limitations. We hope this makes the value of the paper clearer.

Thank you again for the precise references, which will improve the paper. We hope these additions address your main concerns. If they do, we would be grateful if you would consider updating your assessment.

Thank you for the provided response and addressing the mentioned concerns! I have updated by score. It would be interesting to see further development of the method for applied text problems

Reviewer_MQnxOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper revisits distribution/score-matching distillation for continuous diffusion language models. It provides a unified mathematical account of existing approaches and develops two instantiations of reverse-KL-based distribution matching: Simplex-DMD, which uses continuous token relaxations and pathwise gradients, and Reinforce-DMD, which uses categorical sampling with a density-ratio-based REINFORCE estimator.

Using the LangFlow teacher, the paper demonstrates complementary improvements across sampling budgets: Simplex-DMD is particularly effective in the low-NFE regime, whereas Reinforce-DMD becomes competitive at larger budgets. The investigation of training and sampling choices also provides practical insights into adapting distributional distillation to continuous language generation. However, the results establish two regime-specific solutions rather than a single student that performs consistently well across sampling budgets.

Overall, the contribution lies primarily in methodological synthesis, careful adaptation, and empirical advancement, rather than a fundamentally new distillation paradigm. Although many individual techniques build on established ideas, the paper's mathematical exposition and practical findings make it a useful reference for future research. Investigating whether the two methods' complementary strengths can be combined in a single budget-robust student, and extending evaluation beyond generative perplexity-entropy frontiers to downstream language tasks, are natural next steps toward a more broadly effective solution.

Critical strengths
  • The paper does a good job explaining the relationships among distribution/score-matching distillation techniques. It goes beyond a literature survey by clarifying how different objectives and gradient estimators arise from choices of discrepancy and student parameterization. This mathematical organization helps readers distinguish substantive methodological differences from alternative representations or special cases.

  • By adapting these techniques to a more recent and competitive teacher, the paper provides an updated assessment of their effectiveness. The results demonstrate that distributional distillation remains a promising route to efficient continuous diffusion language generation. Evaluating quality-diversity frontiers, rather than generative perplexity alone, is also an important positive aspect of the experimental design. The investigation of stabilization, auxiliary-model training, and sampling choices provides useful guidance for subsequent research. The complementary behavior of the two methods is itself informative: it exposes tradeoffs between continuous relaxation and discrete sampling that are not resolved by their shared distribution-matching formulation. These findings motivate future work on combining their advantages in a more budget-robust student.

Critical weaknesses
  • The empirical conclusions rely primarily on generative perplexity-unigram entropy frontiers for unconditional text generation. These metrics are useful for characterizing predictability and token-level diversity, but they do not directly establish performance on downstream language tasks. Even at matched unigram entropy, models may differ substantially in semantic diversity, long-range coherence, factual consistency, and task usefulness. Broader task-based evaluation would help establish the practical significance of the reported improvements.

  • The two proposed methods perform well in different NFE regions, but the paper does not demonstrate a single student that maintains a competitive quality-diversity frontier across the full sampling-budget range. In particular, Simplex-DMD loses diversity at larger budgets, while Reinforce-DMD is less effective in the few-step regime. The current coverage therefore comes from selecting between two separately trained methods. A deeper explanation of this crossover, together with an investigation of whether their advantages can be combined, would strengthen the contribution.

Questions for authors
  • Does the performance crossover between Simplex-DMD and Reinforce-DMD primarily reflect an intrinsic tradeoff between continuous and categorical student parameterizations, or differences in their optimization and stabilization procedures? Clarifying this distinction would help assess whether their complementary strengths could be combined in a single student that performs well across NFE budgets.

  • The paper provides a thorough and insightful analysis of distribution/score-matching distillation methods, while the proposed algorithms focus on DMD-style reverse-KL matching. Given this broader perspective, have the authors explored other distributional objectives, such as SiD, or additional trajectory-based approaches, such as consistency distillation, using the same LangFlow teacher and comparable training budgets? Such controlled comparisons would help clarify the benefits of the proposed objective and estimator choices, beyond improvements attributable to the updated teacher model.

2 comments
AuthorsOpenReview ↗

Rebuttal 1

Thank you for the careful and constructive review. This first comment gives the new evaluations; the second answers your two questions.

1. Evaluation beyond OWT (W1)

We agree that a Gen PPL–entropy frontier on a single corpus is not enough. We added two datasets.

LM1B (Table A). We use the released LangFlow LM1B teacher and the same protocol as Table 2 of the paper. The takeaway is similar to OWT; e.g., at 4 NFE and matched entropy, S-DMD reaches a Gen PPL of 111, against 361 for its teacher and 328 for D-MMD.

Sudoku (Table B). We measure the solve rate over 100,000 test puzzles. At 1 NFE, S-DMD solves 78.6% of the puzzles, against 69.9% for its teacher. From 2 NFE on, our two students have the highest solve rates of all methods (S-DMD at 2 NFE, R-DMD at 4 and 8 NFE), although all methods are then within two points of each other.

A language task such as TinyGSM would be a relevant next step, but we were unable to run it during the discussion period: there is no public continuous-diffusion teacher for this dataset, and training one was beyond our compute budget. We intend to include it in the final version of the paper.

Table A — LM1B: Gen PPL at matched unigram entropy. A small grey value in parentheses is the entropy reached when a method cannot reach the target. Data: entropy 4.31, Gen PPL 51.7.

Method2 NFE4 NFE8 NFE32 NFE64 NFE128 NFE
Target entropy4.384.344.314.304.304.31
LangFlow (teacher)759361192100.093.391.0
MDLM987449222125109109
FMLM110 (4.20)\color{gray}{\scriptstyle (4.20)}95.5 (4.20)\color{gray}{\scriptstyle (4.20)}83.6 (4.21)\color{gray}{\scriptstyle (4.21)}73.3 (4.24)\color{gray}{\scriptstyle (4.24)}69.3 (4.24)\color{gray}{\scriptstyle (4.24)}57.9 (4.20)\color{gray}{\scriptstyle (4.20)}
D-MMD87332817599.088.285.4
S-DMD (ours)24711173.860.6 (4.29)\color{gray}{\scriptstyle (4.29)}56.5 (4.28)\color{gray}{\scriptstyle (4.28)}56.1 (4.28)\color{gray}{\scriptstyle (4.28)}
R-DMD (ours)59024812379.476.785.8

Table B — Sudoku: solve rate (%) on 100k test puzzles.

TeacherMethod1 NFE2 NFE4 NFE8 NFE
MDLMMDLM (teacher)84.0696.9498.4799.12
MDLMD-MMD89.3998.2198.9799.44
MDLMIDLM89.5098.2298.9799.44
LangFlowLangFlow (teacher)69.8997.3099.6199.89
LangFlowS-DMD (ours)78.5598.5399.6799.79
LangFlowR-DMD (ours)72.1897.6099.6899.90
AuthorsOpenReview ↗

Rebuttal 2

2. Why the two methods behave differently across budgets (W2, Q1)

You asked whether the difference comes from the parameterization itself or from optimization and stabilization. In our understanding the two are linked: each parameterization comes with its own way of training.

In our experiments, R-DMD requires the teacher KL anchor for stable training (Eq. (13), Figure 2), but this constraint may also limit its few-step gains. Unregularized S-DMD achieves larger few-step improvements on OWT, while losing unigram entropy at larger sampling budgets (Table 2).

Our interpretation is that the KL anchor contributes to the crossover: it helps preserve diversity, while potentially restricting few-step improvements. Training S-DMD with the same anchor would directly test this interpretation. We have not yet run this ablation and aim to include it in the revised paper.

The pattern also varies across datasets. On LM1B (128-token sequences), S-DMD has lower Gen PPL than R-DMD at every reported budget, while retaining entropy close to the data: 4.28 versus 4.31 at 128 NFE (Table A in Rebuttal 1). The entropy loss is much stronger on OWT, which uses 1,024-token sequences.

We agree that a single student performing at every budget is a natural goal. We think it requires a dedicated methodological study that goes beyond the scope of this paper, and we hope the present work prepares the ground for it.

3. Other objectives with the same teacher (Q2)

Yes, for one objective. We trained a posterior-matching objective (PMD) on OWT, with the same LangFlow teacher and the same training budget as our methods. It corresponds to the posterior ff-divergence of Table 1 of the paper, in the special case of the KL divergence. Since teacher and budget are identical, the comparison isolates the effect of the objective (Table C).

At few steps, PMD stays close to the teacher (322 against 317 at 4 NFE), while S-DMD reaches 45.6. This shows that the distillation improvements of our method are not only due to the LangFlow teacher.

We have not run SiD itself or consistency distillation on the LangFlow teacher due to time constraints in this rebuttal. We agree that these comparisons would be valuable, and we will add them in the revised version of the paper.

Table C — OWT: Gen PPL at matched unigram entropy. Same protocol as Table A Rebuttal 1: a small grey value in parentheses is the entropy reached when a method cannot reach the target.

Method2 NFE4 NFE8 NFE16 NFE128 NFE256 NFE
Target entropy5.655.445.385.205.275.00
LangFlow (teacher)107731714758.639.121.3
PMD112332211751.19.57 (4.62)\color{gray}{\scriptstyle (4.62)}6.39 (4.21)\color{gray}{\scriptstyle (4.21)}
S-DMD (ours)17845.626.717.03.60 (3.82)\color{gray}{\scriptstyle (3.82)}2.51 (3.31)\color{gray}{\scriptstyle (3.31)}
R-DMD (ours)111255634768.826.914.9

Thank you again. We hope this is useful for your assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.