acceptodds
Under review as a conference paper at ICLR 2027

Representation-Space MMD for Diffusion Language Models

Abstract

We introduce a post-training method for diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM. To estimate MMD, we retain contextual features at individual token positions, obtaining multiple observations per sequence from a single extractor pass. We optimize this objective using policy gradients for discrete models and direct differentiation through generated latents for continuous models. In both cases, computing the loss directly from these features enables efficient post-training without full sampling trajectories or jointly trained auxiliary models. Experiments show lower generative perplexity at comparable entropy on OpenWebText and better accuracy–computation trade-offs on GSM8K. Applied to 16B DMax-LLaDA2.0 models with hybrid masked–uniform diffusion, the method increases their decoding parallelism while providing similar or higher accuracy on math and code benchmarks.

Primary area:
generative models
Keywords:
diffusion language models, post-training, few-step models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_3QLAOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a post-training procedure for DLMs, with the goal of improving performance vs. number of steps tradeoff (improve performance with fewer generation steps) which requires teaching the model to decode more tokens per forward pass. The main motivation is to simplify DLM postraining as compared to two families of methods discriber in ll.031-037 (distillation with teacher-derived targets) while matching them on quality at given step budget.

What proposed method does roughly is:

  • does sample positions S and then sample completions at these positions by the trainable model, G*B times for some context c
  • then get representations of these positions with a pre-trained frozen MDM
  • also get representations for these positions for the ground truth for that context c
  • then compute MMD loss using RBF as a similarity
  • training with MMD as reward (per group, paper uses REINFORCE since to calculate gradients w.r.t. params) loss encourages sampling examples that tend to be more similar in their representations to ground truth representations and less similar within themselves (within B sampled examples), these are also examples that are more likely under the data distribution. Intuitively, given the current partially-masked input, choose per-position distributions such that filling all masked positions at once, independently, tends to produce sequences a pret-trained DLM would recognize as real text.

The continuous variant differentiates through generated representations instead of REINFORCE.

Critical strengths

The soundness of the proposed method constitutes the main strength. The method is carefully designed, main design choices are ablated, the idea of using MMD in the frozen feature space of a pre-trained DLM is novel to my knowledge, simplifies the post-raining design and is shown to work in the experiments.

Critical weaknesses

None

Questions for authors
  • how does the quality of a pre-trained DLM in who's feature space we perform MMD effects the method performance? Does it need to be the same checkpoint as the trained model?
  • in the current version only representations from a single layer of the frozen reference DLM are used, how are the layer(s) selected? Did authors observe layer choice impacting the learning quality?
  • how are the hyperparameters like \sigma and the layer index chosen?
  • the main motivation is to simplify DLM post-training as compared to two families of methods described in ll. 032-037. However, if I am not mistaken, for the discrete DLM experiments only baselines from the latter family are used (, i.e. all baselines IDLM, IDLM-REINFORCE, DiDi-Instruc all belong to the second family). I think adding baselines from the first family (SDTT , CDLM) would make the comparison complete.
  • Why no other baselines are included in Table 1 (IDLM or DiDi-Instruct style methods)?
4 comments
AuthorsOpenReview ↗

Rebuttal | Part 1

We thank the reviewer for their positive and constructive feedback. We address each question below and will provide a revised manuscript and a general response to the reviews in the coming days.


Q1. Feature extractor quality and choice of checkpoint

how does the quality of a pre-trained DLM in who's feature space we perform MMD effects the method performance? Does it need to be the same checkpoint as the trained model?

We thank the reviewer for the interesting question. To explore it, we repeat MMD post-training on TinyGSM with features from a different pretrained DLM.

ELF. We use ELF-B and the larger, more accurate ELF-M (Appendix C.6) as feature extractors for both models:

ModelFeature extractor4 steps8 steps16 steps32 steps64 steps
ELF-B-MMDELF-B (own checkpoint)14.227.534.234.335.2
ELF-B-MMDELF-M11.925.132.734.336.0
ELF-M-MMDELF-M (own checkpoint)22.235.541.443.243.6
ELF-M-MMDELF-B22.835.041.142.443.0

The stronger ELF-M features do not improve ELF-B, and the weaker ELF-B features work almost as well for ELF-M as its own features.

MDLM. We could not find a discrete DLM trained on TinyGSM with noticeably better accuracy. Therefore, we use MDLM 1.7B [1], a much larger general-purpose model that was not trained on TinyGSM:

ModelFeature extractor8 steps16 steps32 steps48 steps
MDLM–25.335.236.735.5
MDLM-MMDMDLM (own checkpoint)27.040.348.252.3
MDLM-MMDMDLM 1.7B [1]23.938.047.250.8

With MDLM 1.7B features, the accuracy is slightly lower than with the model's own features, which we attribute to MDLM 1.7B not being fitted to the TinyGSM format. MMD post-training still clearly improves over the base MDLM at 16–48 steps.

Overall, the feature extractor does not need to be the same checkpoint as the trained model: MMD post-training also works with features from a different model, even one not trained on the target dataset. However, a larger feature extractor does not necessarily bring additional gains, so we suggest using the pretrained model itself by default, since it also requires no additional model.

[1] Sahoo et al. (2026), “Scaling Beyond Masked Diffusion Language Models”


Q2. Selection of the feature extraction layer

How are the layer(s) selected? Did authors observe layer choice impacting the learning quality?

By default, we use a single middle layer for simplicity. The only exception is the unconditional MDLM setup, where an earlier layer gave slightly better results.

We also tried other layers and did not find the particular choice to be critical. Our rule of thumb is to take a layer that is deep enough to provide a semantically structured feature space. Below, we compare different layers for MDLM and ELF on TinyGSM:

Layer (MDLM)8 steps16 steps32 steps48 stepsLayer (ELF)2 steps4 steps8 steps16 steps32 steps64 steps
425.838.347.850.933.712.324.833.535.136.3
7 (ours)27.040.348.252.36 (ours)4.014.227.534.234.335.2
1024.738.347.852.493.512.225.332.635.636.5

Overall, the layer choice does not significantly impact the learning quality. The middle layer works best at smaller step budgets in both setups, and no other layer consistently outperforms it, so we suggest using it by default.


Q3. Selection of σ and the layer index

How are the hyperparameters like \sigma and the layer index chosen?

σ selection. We first find a reasonable range of values with the median heuristic from prior work [2, 3]: we extract token features from a batch of real training samples and compute the median pairwise distance between them. We then try a few values below and above this estimate and choose the one that performs best on a held-out set. This set contains 512 examples from the GSM8K training split and does not overlap with the training or test data.

We did not perform any extensive σ tuning, and the results are not too sensitive to the particular value within a reasonable range. The σ ablation is provided in our response to Reviewer j8mG (Q4).

Layer selection. As discussed in Q2, we use the middle layer by default and do not tune this choice in most cases.

[2] Gretton et al. (2012), “A Kernel Two-Sample Test”

[3] Garreau et al. (2017), “Large Sample Analysis of the Median Heuristic”

AuthorsOpenReview ↗

Rebuttal | Part 2


Q4. Baselines from the first family of methods: SDTT and CDLM

I think adding baselines from the first family (SDTT, CDLM) would make the comparison complete.

We thank the reviewer for the suggestion and have added both baselines.

SDTT (OWT, unconditional). At matched entropy, SDTT has the highest generative perplexity among all methods at 8, 16 and 32 steps, please see the figure: anonymous link

CDLM (TinyGSM). We adapted the CDLM implementation to our setup:

Model4 steps8 steps16 steps32 steps48 steps
MDLM2.425.335.236.735.5
CDLM9.028.335.338.038.1
MDLM-MMD5.127.040.348.252.3
MDLM-MMD+CDLM10.337.849.953.655.1

CDLM is more accurate at 4–8 steps, while MDLM-MMD is 5–14 points better at 16–48 steps.

As a side note, our conclusion mentions combining MMD with distillation losses as a promising direction. Interestingly, when we applied CDLM on top of MDLM-MMD, it significantly improved accuracy at every budget. At 8–48 steps, the gain is 2.8–10.8 points, while CDLM on the base MDLM gains at most 3 points. These results encourage further work on such combinations.

CDLM results for the 16B DMax setup are reported under Q5 below.


Q5. Additional baselines in Table 1

Why no other baselines are included in Table 1 (IDLM or DiDi-Instruct style methods)?

We initially could not obtain improvements with IDLM at the 16B scale and therefore did not include it. During the rebuttal, we found a working configuration and report it below, together with CDLM. We focus on IDLM rather than DiDi-Instruct, since IDLM is a more recent approach that shows superior results on other benchmarks.

MethodGSM8KMATH500Minerva-AlgebraASDIVHumanEval-InstructMBPP-Instruct
DMax92.1 / 5.4875.4 / 5.9491.5 / 7.0392.5 / 5.6283.5 / 7.3679.2 / 5.86
DMax-CDLM92.0 / 5.6675.3 / 6.2491.4 / 7.3692.9 / 5.7483.9 / 7.6081.0 / 5.86
DMax-IDLM92.1 / 6.1676.3 / 6.8691.1 / 8.1492.7 / 6.2486.3 / 8.2081.6 / 6.13
DMax-MMD92.1 / 6.1576.0 / 6.8492.1 / 8.1992.9 / 6.2085.9 / 8.0783.0 / 6.10

DMax-IDLM is close to DMax-MMD in TPF and in accuracy on four of the six benchmarks, with more noticeable DMax-MMD gains on Minerva-Algebra (+1.0) and MBPP (+1.4). Also, DMax-IDLM takes about 50% more training time (~20 min vs ~13 min for DMax-MMD) and requires training an additional model.

DMax-CDLM improves TPF only slightly over DMax (by at most 0.33). Compared to DMax-MMD, it has lower TPF on every benchmark and lower accuracy on five of the six.

AuthorsOpenReview ↗

Rebuttal Follow-up

Dear Reviewer 3QLA,

Thank you for your positive assessment and helpful suggestions. Since the discussion ends soon, we wanted to follow up on our response.

In our rebuttal, we answered your questions about the choice of the feature extractor, its layer and the kernel bandwidth with additional experiments, and added the baselines you asked about: SDTT on OpenWebText, CDLM on TinyGSM, and IDLM and CDLM for the 16B DMax models in Table 1.

We have added all these results to the revised paper. Please let us know if you have any remaining questions.

I thank the authors for their extensive responses. I will keep my positive score for this paper.

Reviewer_wspZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a post-training method for diffusion language models that uses MMD to match generated and reference samples in a frozen model’s token feature space. It supports both discrete and continuous models. Experiments show lower generative perplexity and better accuracy–computation trade-offs, including on 16B models.

Critical strengths
  1. This method is simple yet broadly applicable across different types of diffusion language model.
  2. Experiments across several model types and tasks, together with the ablations, provide clear evidence for the main design choices.
Critical weaknesses
  1. The 16B experiments lack a baseline that continues training with the original DMax objective under a comparable budget. This makes the added benefit of MMD over further standard training unclear.
  2. The paper reports the chosen values of BB and σ\sigma, but does not clearly explain how they were selected or systematically test sensitivity to them. This leaves the method's robustness unclear.
Questions for authors
  1. How does MMD compare with continuing training using the original DMax objective on the same data and with a comparable training budget?
  2. How were BB and σ\sigma selected, and how sensitive is performance to these choices?
3 comments

We thank the reviewer for their feedback. Below, we address each concern in detail. We will also provide a revised manuscript and a general response in the coming days.


Q1/W1. MMD versus continuing DMax training

How does MMD compare with continuing training using the original DMax objective on the same data and with a comparable training budget?

The original DMax was trained for 350,000–500,000 steps, while our MMD post-training uses only 400 steps (<0.5% of the total training time), including all additional costs. We continued DMax training with the original objective for 1,000 more steps, which even exceeds the MMD training time.

MethodGSM8KMATH500Minerva-AlgebraASDIVHumanEval-InstructMBPP-Instruct
DMax92.1 / 5.4875.4 / 5.9491.5 / 7.0392.5 / 5.6283.5 / 7.3679.2 / 5.86
DMax + Continued Training (CE)91.4 / 5.4074.6 / 5.8092.2 / 6.8892.6 / 5.4882.9 / 7.2780.1 / 5.91
DMax-MMD92.1 / 6.1576.0 / 6.8492.1 / 8.1992.9 / 6.2085.9 / 8.0783.0 / 6.10

Continuing training keeps both accuracy and TPF close to the original DMax. In other words, the MMD gains come from the objective itself rather than from additional training.


Q2/W2. Selection of B and σ and performance sensitivity

The paper reports the chosen values of B and σ, but does not clearly explain how they were selected or systematically test sensitivity to them.

Selection of B. Those values were automatically selected based on the performance on the held-out set.

Below, we show the results are close across B values for both ELF-MMD and MDLM-MMD. Conlusion: the results are not sensitive to B.

B (ELF-MMD)4 steps8 steps16 steps32 steps64 steps
214.428.033.734.535.2
414.428.033.734.935.2
814.527.433.834.535.2
64 (ours)14.227.534.234.335.2
B (MDLM-MMD)8 steps16 steps32 steps48 steps
2 (ours)27.040.348.252.3
426.840.048.052.2
826.240.348.251.7

RBF bandwidth (σ) ablation. We first find a reasonable range of values with the median heuristic from prior work [1, 2]: we extract token features from a batch of real training samples and compute the median pairwise distance between them.

Then, we tried a few values below and above this estimate and chose the one that performed best on a held-out set. This set contains 512 examples from the GSM8K training split and does not overlap with the training or test data.

σ (MDLM-MMD)8 steps16 steps32 steps48 stepsσ (ELF-MMD)4 steps8 steps16 steps32 steps64 steps
6025.940.248.951.9200013.126.333.035.035.7
90 (ours)27.040.348.252.3300013.126.133.234.335.2
11025.338.847.952.53330 (ours)14.227.534.234.335.2
16024.636.847.051.9400012.725.632.834.335.9
22022.737.246.951.8500012.424.832.434.335.4

The results are not too sensitive to the particular σ value, provided it is not too small or too large.


[1] Gretton et al. (2012), “A Kernel Two-Sample Test”

[2] Garreau et al. (2017), “Large Sample Analysis of the Median Heuristic”

AuthorsOpenReview ↗

Rebuttal Follow-up

Dear Reviewer wspZ,

Thank you again for your time and helpful comments. As the discussion period closes tomorrow, we would like to briefly follow up on our response.

In our rebuttal, we addressed your concerns about continued training and about hyperparameter selection and sensitivity through additional control experiments and ablations across different DLM families.

These new results are now included in the revised paper. We would appreciate your thoughts on whether these additions resolve your concerns, and we would be glad to clarify any remaining questions.

Dear Authors,

Thank you for your rebuttal and the corresponding revisions. I have carefully reviewed your responses, and I am satisfied that the concerns I raised have been adequately addressed. Based on the clarifications and changes provided, I am willing to adjust my score from 2 to 3, as I now consider the paper to be at a marginally acceptable level. I wish the authors the best with the final version of the paper.

Sincerely, Reviewer wspZ

Reviewer_j8mGOpenReview ↗
Rating
2(+1)
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a post-training method for diffusion language models that minimizes MMD between generated and reference distributions in the contextual token feature space of a frozen pretrained DLM. The method retains token-level representations and supports both discrete models, optimized through policy gradients, and continuous models, optimized through direct differentiation. Experiments on text generation, mathematical reasoning, and code generation show improved quality-computation trade-offs across several settings.

Critical strengths
  1. The approach accommodates both discrete and continuous DLMs. The MMD training stage requires neither full teacher sampling trajectories nor jointly trained auxiliary models.

  2. Contextual token features provide multiple observations per sequence while retaining contextual information. The generated-generated term discourages excessive concentration, and the estimator explicitly accounts for dependencies among features within a sequence.

Critical weaknesses
  1. The paper provides a sound construction of the MMD estimator, but the connection between matching contextual token-feature distributions and improving the final sequence distribution remains largely empirical. As acknowledged by the authors, the cited injectivity results do not establish identifiability for the proposed objective. Moreover, the discrete formulation optimizes samples from a single factorized denoiser pass, whereas evaluation uses iterative decoding. The paper could better explain why this surrogate specifically improves generation under limited sampling budgets.

  2. The loss ablations and IDLM-REINFORCE comparison are informative, but some important controls are missing. In particular, the 16B experiments compare MMD-post-trained models against their original checkpoints without a matched-budget continuation using the original training objective. For continuous models, the overall recipe also includes bootstrapping and decoder fine-tuning, with optional IRD. Existing ablations partially separate these contributions, but do not fully isolate the gains attributable to MMD over continued training and the accompanying modifications.

  3. The reported training times and accuracy–TPF curves provide useful evidence, but do not establish a comprehensive computational advantage over competing methods. Multiple generated samples, frozen-feature extraction, bootstrapping, and optional IRD all contribute to training cost. In addition, Section 3.3 specifies an extra decoding evaluation after refinement, while the experimental reporting does not consistently clarify whether “steps” includes this evaluation. This distinction matters particularly at small sampling budgets.

Questions for authors
  1. Can the authors provide evidence connecting improvements in the proposed MMD objective to improvements in final generation quality across decoding budgets? In particular, can they distinguish benefits from contextual distribution matching from changes in prediction confidence or calibration under confidence-based decoding?

  2. How does MMD compare with continuing training under the original objective using the same data and a comparable computational budget, particularly for the 16B models? For ELF, how much of the improvement remains after controlling for bootstrapping and decoder fine-tuning?

  3. Table 6 uses substantially different MMD batch sizes across settings, including (B=64) for conditional ELF training versus (B=2) or (4) for unconditional generation. What motivates this difference, and how does reducing (B) affect training stability and downstream performance in the conditional setting?

  4. How were the feature extraction layer and RBF bandwidth selected for each setting? Were these choices made using held-out validation data, and how sensitive are the results to them?

8 comments
AuthorsOpenReview ↗

Rebuttal | Part 1

We thank the reviewer for their feedback. We respond to each concern and question below. A revised manuscript and a general response will follow in the coming days.


W1.1 | Theoretical grounding of matching token-feature distributions

The paper provides a sound construction of the MMD estimator, but the connection between matching contextual token-feature distributions and improving the final sequence distribution remains largely empirical. As acknowledged by the authors, the cited injectivity results do not establish identifiability for the proposed objective.

We agree that the cited results [1,2] do not by themselves establish identifiability for our token-level objective. For token sequences, the missing piece is one assumption: each contextual token feature uniquely determines the sequence it comes from. Under this assumption, equal token-feature distributions imply equal sequence distributions, so with a characteristic kernel, zero population MMD is reached only at the target distribution [1].

We find this assumption plausible. [2] proves a similar property for the last-token state of causal transformers, and [3] formulates it for every layer. Our feature extractors are bidirectional, so every token feature depends on the whole sequence. Moreover, full self-attention can realize maps in which each token's output identifies the entire input sequence, known as contextual mappings [4]. We will state the assumption explicitly in the revision and add this discussion.


W1.2 | Single-pass factorized training and few-step iterative decoding

The discrete formulation optimizes samples from a single factorized denoiser pass, whereas evaluation uses iterative decoding. The paper could better explain why this surrogate specifically improves generation under limited sampling budgets.

We thank the reviewer for raising this important question.

How does feature-space MMD handle factorized outputs?

Cross-entropy fits the token marginals at each position separately, so sampling many positions independently can mix tokens from different plausible completions, producing wrong combinations.

Our loss instead evaluates whole sampled sequences through contextual features that capture how their tokens fit together. This feedback can concentrate the probabilities at different positions on mutually compatible tokens, making incompatible combinations less likely even though sampling remains independent. The generated–generated term counterbalances this concentration by discouraging collapse to a single output.

To sum up, factorization limits what one pass can represent, but distribution matching still can improve samples within this family, as shown by prior work [5,6] and our experiments.

Why does single-pass training help few-step decoding?

As in [5,6], our training covers all noise levels. Therefore, at each decoding step, more coherent completions should provide better context for later steps. This can improve iterative decoding without training through entire sampling trajectories.

We will add this discussion to the revision.


[1] Li et al. (2017), “MMD GAN: Towards Deeper Understanding of Moment Matching Network”

[2] Nikolaou et al. (2026), “Language Models are Injective and Hence Invertible”

[3] von Strauss (2025), “Transformer Injectivity & Geometric Robustness - Analytic Margins and Bi-Lipschitz Uniformity of Sequence-Level Hidden States”

[4] Yun et al. (2020), “Are Transformers universal approximators of sequence-to-sequence functions?”

[5] Li et al. (2026), “IDLM: Inverse-distilled Diffusion Language Models”

[6] Hoogeboom et al. (2026), “Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD”

AuthorsOpenReview ↗

Rebuttal | Part 2


Q1 | MMD objective vs prediction confidence under confidence-based decoding

Can the authors provide evidence connecting improvements in the proposed MMD objective to improvements in final generation quality across decoding budgets? In particular, can they distinguish benefits from contextual distribution matching from changes in prediction confidence or calibration under confidence-based decoding?

We thank the reviewer for this interesting question.

First, our OWT experiments already show improvements under ancestral sampling (random reveal positions) across different temperatures.

For TinyGSM, we additionally evaluate ancestral sampling with greedy token predictions (T=0). Positions are revealed at random and tokens are taken by argmax, so prediction confidence plays no role. MMD still improves accuracy at every step budget.

Model4 steps8 steps16 steps32 steps48 steps64 steps
MDLM1.85.714.122.326.128.9
IDLM1.76.614.520.022.924.9
MDLM-MMD3.28.417.725.529.330.5

Also, we evaluate two confidence-only variants:

  • Hybrid MDLM: the base MDLM predicts the tokens and MDLM-MMD's maximum probabilities select the positions to reveal at each step.
  • MDLM-Conf-MMD: the base MDLM predicts the tokens, while an MLP confidence head on top of the frozen MDLM, trained with MMD, selects the positions.
GSM8K test accuracy (%)4 steps8 steps16 steps32 steps64 steps
MDLM2.425.335.236.734.0
Hybrid MDLM1.517.533.644.952.6
MDLM-Conf-MMD1.624.538.146.850.2
MDLM-MMD5.127.040.348.254.0

We observe that both token predictions and reveal decisions contribute to the gains. The confidence-only variants improve MDLM mainly at larger budgets, while the full MDLM-MMD is the best at every budget. Together with the ancestral results, the gains cannot be explained solely by changes in confidence-based reveal ordering.

We will add this analysis to the revision.


Q2/W2. | MMD vs continuing training

How does MMD compare with continuing training under the original objective using the same data and a comparable computational budget, particularly for the 16B models?

We ran this comparison in three setups. In each case, continuing training with the original objective and same data does not reproduce the gains of MMD.


16B DMax. The original DMax was trained for 350,000–500,000 steps, while our MMD post-training uses only 400 steps (<0.5% of the total training time), including all additional costs. We continued DMax training with the original objective for 1,000 more steps, which even exceeds the MMD training time.

MethodGSM8KMATH500Minerva-AlgebraASDIVHumanEval-InstructMBPP-Instruct
DMax92.1 / 5.4875.4 / 5.9491.5 / 7.0392.5 / 5.6283.5 / 7.3679.2 / 5.86
DMax + Continued Training (CE)91.4 / 5.4074.6 / 5.8092.2 / 6.8892.6 / 5.4882.9 / 7.2780.1 / 5.91
DMax-MMD92.1 / 6.1576.0 / 6.8492.1 / 8.1992.9 / 6.2085.9 / 8.0783.0 / 6.10

Continuing training keeps both accuracy and TPF close to the original DMax.


MDLM (TinyGSM). The original MDLM was trained for 250,000 steps, while MMD post-training takes 8,000 steps (~4% of the full training time, including all additional costs). We continued MDLM training for 12,000 steps, about 1.13× the MMD compute:

Model8 steps16 steps32 steps48 steps
MDLM25.335.236.735.5
MDLM + Continued Training (CE)24.433.235.333.3
MDLM-MMD27.040.348.252.3

The conclusion is the same: continuing training did not improve the base MDLM model.


ELF (TinyGSM). The original ELF was trained for 500,000 steps, while MMD post-training takes 7,000 steps (<1% of the full training time, including all additional costs), plus 20,000 steps for the optional IRD stage. We continued ELF training for 25,000 steps, i.e., 5% of the original training.

Model2 steps4 steps8 steps16 steps32 steps64 steps
ELF2.06.013.823.328.831.6
ELF + Continuing Training1.75.913.723.029.232.1
ELF-MMD4.014.227.534.234.335.2

Again, continuing training kept the accuracy close to the base ELF.

AuthorsOpenReview ↗

Rebuttal | Part 3


Q2/W2. | For ELF, how much of the improvement remains after controlling for bootstrapping and decoder fine-tuning?


Bootstrapping. Appendix C.4 already ablates the number of bootstrapping steps. We repeat the TinyGSM results below: bootstrapping provides only modest improvements.

Bootstrapping steps2 steps4 steps8 steps16 steps
03.611.925.033.0
14.013.626.533.3
24.014.127.434.1
33.913.927.433.7
44.014.227.534.2

Decoder fine-tuning. We note that all ELF-based approaches include the decoder objective. Therefore, the decoder loss is simply adopted from ELF, as stated in Appendix B.2.

For completeness, we also trained ELF-MMD without it:

Decoder finetuning4 steps8 steps16 steps32 steps64 steps
With14.227.534.234.335.2
Without13.927.533.734.435.4

Accuracy changes by at most 0.5 points, so decoder fine-tuning can be turned off if needed.


W3 | Training cost

Multiple generated samples, frozen-feature extraction, bootstrapping, and optional IRD all contribute to training cost.

For transparency, we report the exact training costs of MDLM, ELF and their few-step baselines on TinyGSM. All timings were measured on H100 GPUs and include all additional costs of each method. ELF-PD has six training stages with different times per step, so we report only its total cost.

Model (TinyGSM)Seconds/stepBatch / GPUGPU hours
ELF0.2164234.88
ELF-GAN1.13649.40
ELF-PD-64708.64
ELF-MMD0.35641.36
ELF-MMD (IRD stage)0.89649.84
Model (TinyGSM)Seconds/stepBatch / GPUGPU hours
MDLM0.3064168.19
IDLM1.25649.76
DiDi-Instruct0.2713.00
CDLM0.746416.60
MDLM-MMD0.40647.11

Overall, the overhead of MMD is small. A single MMD step is somewhat more expensive than a step of the original training, but post-training is short, so its total cost is only a tiny fraction of the base model training.

MMD is also cheap compared with other methods. ELF-MMD is much cheaper than ELF-GAN and ELF-PD, and with the optional IRD stage its cost is comparable to ELF-GAN. MDLM-MMD is cheaper than IDLM and CDLM, and only DiDi-Instruct requires less compute due to their single batch size setup.


W3 | Extra decoding step for ELF

Section 3.3 specifies an extra decoding evaluation after refinement, while the experimental reporting does not consistently clarify whether “steps” includes this evaluation.

Importantly, all ELF-based approaches apply this decoding step. Since all our ELF-MMD comparisons are within the same ELF family, we did not include this step in those comparisons.

However, we overlooked this when comparing with FMLM+. Below, we provide the corrected comparison, where the NFE of the ELF-based models includes the decoding step:

Model3 NFE5 NFE9 NFE17 NFE33 NFE
FMLM+3.19.017.028.731.2
ELF-MMD4.014.227.534.234.3
ELF-MMD+IRD7.520.832.535.636.0

ELF-MMD still significantly outperforms FMLM+.

In the revision, we will update the comparison with FMLM+ and clarify in Section 3.3 that this step is applied to all ELF models.


Q3. MMD batch size in conditional and unconditional settings

Table 6 uses substantially different MMD batch sizes across settings, including (B=64) for conditional ELF training versus (B=2) or (4) for unconditional generation. What motivates this difference, and how does reducing (B) affect training stability and downstream perform in the conditional setting?

We apologize for the confusion. We selected B=64 for conditional ELF on the held-out set, but in fact the results are not sensitive to B. Reducing B also does not affect training stability. Also, note that we keep the total generated-sample batch size fixed across B values, therefore it does not affect the training time.

Below, we show the results are close across B values for both ELF-MMD and MDLM-MMD.

B (ELF-MMD)4 steps8 steps16 steps32 steps64 steps
214.428.033.734.535.2
414.428.033.734.935.2
814.527.433.834.535.2
64 (ours)14.227.534.234.335.2
B (MDLM-MMD)8 steps16 steps32 steps48 steps
2 (ours)27.040.348.252.3
426.840.048.052.2
826.240.348.251.7

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response and revision

We sincerely thank the reviewers for their constructive feedback and helpful suggestions. We have revised the paper to address the main concerns below and highlighted the corresponding changes in blue\text{\color{blue}{blue}}. We welcome further questions and continued discussion, and would be happy to clarify any remaining concerns.


(i) MMD vs continued training [j8mG\color{firebrick}{\textbf{j8mG}}, wspZ\color{firebrick}{\textbf{wspZ}}]: In App. I, we added continued-training controls for DMax, MDLM, and ELF using their original objectives and the same data. Continued training with comparable or larger budgets does not improve the base models, so the gains from MMD remain the same.

(ii) Distinguishing better token predictions from more accurate reveal decisions [j8mG\color{firebrick}{\textbf{j8mG}}]: In App. J, we report random-reveal decoding results at T=0T=0 and two controls that modify reveal decisions while retaining the base model’s token predictions. Both components contribute: changes in confidence predictions do not explain the full gains.

(iii) Theoretical scope and why distribution matching can benefit over factorized predictions [j8mG\color{firebrick}{\textbf{j8mG}}]: In Sec. 3.1, we explicitly state a sufficient assumption under which matching token-feature distributions identifies the sequence distribution, and explain how prior results motivate this assumption. In App. K, we discuss how contextual feedback can improve predictions within a factorized model and why training across noise levels can benefit iterative decoding.

(iv) Hyperparameter sensitivity [j8mG\color{firebrick}{\textbf{j8mG}}, wspZ\color{firebrick}{\textbf{wspZ}}, 3QLA\color{firebrick}{\textbf{3QLA}}]: We expanded App. C with ablations of MMD batch size, RBF bandwidth, and feature-extraction layer. We show that the method is largely insensitive to these choices near the defaults and requires only modest tuning. We also clarify the selection procedure and practical defaults.

(v) Training costs and inference accounting [j8mG\color{firebrick}{\textbf{j8mG}}]: App. H reports measured training costs, including method-specific overheads, showing that MMD post-training takes only a tiny fraction of the base model training time and is cheaper than most competing few-step methods.

We also clarify in Sec. 3.3 and App. B.2 that the additional decoding evaluation and decoder-training objective in the ELF setup are shared by all ELF-based methods and are not specific to our MMD procedure.

(vi) Broader baseline comparisons [3QLA\color{firebrick}{\textbf{3QLA}}]: We added SDTT on OpenWebText, CDLM on TinyGSM, and IDLM and CDLM at the 16B DMax scale (Secs 4.1–4.3 and Tab. 1).

AuthorsOpenReview ↗

Post-Discussion Revision

We are deeply grateful to all reviewers for engaging in the discussion and to Reviewer j8mG\color{firebrick}{\textbf{j8mG}} in particular for clarifying the important comparison we missed in our first response. We added this comparison and further details to the revised paper and highlighted the changes in red\color{red}{\textbf{red}}.

(vii) Held-out MMD and generation quality. App. L shows that checkpoints with lower held-out MMD generally achieve higher accuracy at similar budgets on TinyGSM. The endpoint comparisons extend this conclusion for OWT models.

(viii) Contextual features. In App. C.2, we repeated the MDLM comparison from Figure 6 with random reveal order and greedy token selection. Bidirectional DLM features achieved the highest accuracy at all four budgets, followed by AR features, which use only left context, and then static token embeddings, which contain no context.

(ix) Hyperparameter selection. App. G clarifies the validation procedures and selection criteria for OWT and DMax.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.