acceptodds
Under review as a conference paper at ICLR 2027

Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models

Abstract

Diffusion language models (dLLMs) promise fast inference by generating multiple tokens in parallel, but suffer severe performance degradation when parallel decoding is pushed too aggressively. We introduce Temporal Self-Distillation (TSD), a simple on-policy method that trains dLLMs for fast inference by distilling predictions across time. Specifically, TSD distills the model's denoising distribution at earlier timesteps toward its distribution at the final timestep at which a token is committed. This encourages earlier predictions to better anticipate the model's eventual output, enabling much more aggressive parallel decoding. Because its teacher signal comes from the model itself, TSD requires no offline teacher generation and applies seamlessly to both base and post-trained policies. Across seven benchmarks in mathematics, planning, and code, TSD substantially shifts the speed–quality frontier toward the low-compute regime. TSD thus provides a simple, single-stage approach to accelerating dLLMs, achieving speedups competitive with offline distillation while avoiding a complex two-stage pipeline.

Primary area:
generative models
Keywords:
diffusion models, dllm, self-distillation, fast-inference, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VjmUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Temporal Self-Distillation (TSD), an on-policy method designed to accelerate inference in discrete diffusion language models (dLLMs), with a primary focus on masked diffusion models (MDMs). The key premise is that dLLMs suffer steep performance drops when decoding with low numbers of function evaluations (NFEs) because early-timestep predictions lack sufficient decoded context.

TSD addresses this by distilling the model's predictive distribution at an earlier, masked timestep t^\hat{t} toward its own predictive distribution at the commitment timestep tlt_l. To make training computationally tractable, the authors introduce a single-timestep Monte Carlo approximation and weight the distillation loss using a verifier reward V(x0)V(x^0), augmented with a length-aware penalty to discourage verbosity. The method is evaluated across mathematical reasoning (GSM8K, MATH-500), planning (Countdown, Sudoku), and code generation (HumanEval, MBPP, LiveCodeBench) on top of LLaDA-8B and RL-post-trained checkpoints (GDSD, ESPO), with an extension to uniform-state diffusion

Critical strengths
  • This paper has a clear motivation and addresses an important research question. Targeting the discrepancy between early-stage and commitment-stage token predictions is a natural and well-motivated avenue for enabling more aggressive parallel unmasking in dLLMs.
  • Unlike offline distillation approaches (e.g., dParallel, SDTT) that require caching full teacher trajectories or maintaining separate teacher models, TSD derives its training targets dynamically along the model's own on-policy rollouts.
  • The authors evaluate across diverse domains (code, math, planning), analyze varying canvas lengths and explore compatibility with RL post-training (GDSD, ESPO).
  • The identification and discussion of KL divergence instability versus Jensen–Shannon Divergence (JSD) boundedness provides useful practical insight for distilling discrete distributions.
Critical weaknesses

On comparison with baselines

  • A central methodological flaw in the main benchmark comparisons (Figures 3 and 4) is the baseline fairness. For GSM8K, MATH-500, and Coding (MBPP, HumanEval, LiveCodeBench), TSD fine-tunes LLaDA-8B-Instruct via LoRA on task-specific training sets: GSM8K training split, MATH training split (~7.5k), and KodCode-Light-RL-10k (Table 1). The primary baseline LLaDA (Fast-dLLM), is the off-the-shelf, general-purpose base model without any in-domain adaptation.

On decoding efficiency confounding factors

  • In Fast-dLLM, decoding proceeds in fixed-size blocks (e.g., block length 32). In this setup, the total number of forward passes (NFE) depends strongly on the total sequence length: shorter generations require fewer total blocks to unmask, which mechanically lowers the NFE. The authors introduce Length-Aware Reward Weighting (Eq. 9), which explicitly penalizes completions exceeding a hand-picked target length LtargetL_{\textit{target}}. The ablation in Figure 6 and Figure 18 reveals that without length-aware weighting (raw-reward), the low-NFE advantage degrades considerably, particularly at larger canvas sizes. This brings a question of correctness: is TSD actually teaching the model to unmask more tokens in parallel at each step, or is it simply penalizing verbosity so that the model produces shorter answers that finish in fewer blocks?

On over confidence and peak degradation

  • On MATH-500 (Figure 4), the base model achieves 35.4% accuracy at higher NFEs, whereas TSD tops out at 31.8%, a drop of 3.6 percentage points. Similarly, on LiveCodeBench (Figure 3), the base model surpasses TSD at higher NFEs.
  • As shown in Appendix C.8 (Figures 20–22), increasing the number of Monte Carlo timesteps from K=1 to K=6 accelerates NFE reduction but causes accuracy to drop sharply after just 1,000 steps. The authors state on lines 278–282 that prolonged training overfits to verifier rollouts and erodes general reasoning.
  • This indicates that TSD induces artificial probability sharpening (self-reinforcing overconfidence), which forces Fast-dLLM to prematurely commit tokens above threshold λ\lambda. If the model cannot safely sustain training beyond 1,000–2,000 steps without degrading, the method acts more as an early-stopping heuristic than a stable distillation procedure.
Questions for authors
  • Controlled baseline
  • Debiasing length from parallelism
  • Generalization to non verifiable domains
2 comments

We thank the reviewer for recognizing the clear motivation, the fact that the on-policy design avoids offline teacher trajectories and the breadth of the evaluation and ablations. We address each point below. Changes in the revised PDF are marked in red.

W1 & Q1 (baseline fairness, controlled baseline)

We agree that in-domain training must be controlled for. We trained a rejection-sampling fine-tuning (RFT) control with exactly TSD's training setup (GSM8K data, prompts, on-policy rollouts, verifier, length-aware weighting, hyperparameters). The only difference is the loss: RFT uses the standard masked-diffusion loss on the weighted rollouts. Results on the full GSM8K test set (L=256, Fast-dLLM):

λ\lambdaModelAcc. (%)NFE ↓Answer lengthTokens/NFE ↑
0.99LLaDA80.7126.22341.85
0.99RFT, step 100076.9114.22201.92
0.99RFT, step 200072.3103.12242.17
0.99TSD, step 200079.834.31363.97
0.9LLaDA80.382.12342.85
0.9RFT, step 200073.562.42253.60
0.9TSD, step 200073.221.51376.38
  • The TSD objective drives the speedup. At λ=0.99\lambda=0.99, RFT reduces NFE by only 1.1–1.2× and loses 3.8–8.4 points of accuracy, while TSD reduces NFE by 3.7× at near-base accuracy (79.8% vs. 80.7%). At matched accuracy (λ=0.9\lambda=0.9: TSD 73.2%, RFT 73.5%), TSD needs 2.9× fewer forward passes (21.5 vs. 62.4).
  • Efficiency goes beyond shorter answers. With the same length-aware weighting, RFT's answers barely shorten through step 2000 (224 vs. 234 tokens), and TSD commits 1.8× more tokens per pass (3.97 vs. 2.17). W2 separates the two effects.

This agrees with evidence already in the paper. On Countdown, GDSD was already RL-trained on the same task, yet applying TSD to it still reduces NFE by up to 4.5× (App. C.2). SDTT, trained on the same in-domain corpora, gives no NFE reduction (App. C.3). The full RFT results are in App. C.10; due to compute limits, this control was run on GSM8K only.

W2 & Q2 (shorter answers or more parallelism)

In Fast-dLLM, a shorter answer leaves fewer blocks with content, and blocks holding only end-of-text tokens are cheap but still cost at least one pass. We therefore report two measures that do not depend on length: tokens per forward pass (TPF = answer tokens / NFE) and NFE per answer block (passes spent on blocks containing answer tokens, divided by the number of such blocks). All results are at λ=0.99\lambda=0.99, L=256.

1. Without length weighting, TSD keeps the answer length but still decodes much faster.

TaskModelNFE ↓Answer lengthTPF ↑NFE per answer block ↓
MBPPLLaDA121.12161.7816.9
TSD, raw reward59.62213.718.2
TSD, length-aware37.71223.247.9
CountdownGDSD103.11231.1922.0
TSD, raw reward19.81135.683.8

On MBPP, raw-reward TSD (γlen=0\gamma_{\text{len}}=0) writes answers as long as the base model's (221 vs. 216 tokens), yet halves NFE and doubles TPF. On Countdown, its answers are only 9% shorter, but NFE drops 5.2×. Its cost per answer block is about the same as with length weighting (8.2 vs. 7.9 on MBPP).

2. At equal answer length, TSD needs about 2× fewer passes. Mean NFE on MBPP, grouped by answer length (tokens):

Model97–128129–160161–192193–224225–256
LLaDA73.289.7101.4109.0139.5
TSD, raw reward34.739.551.558.666.5
TSD, length-aware38.048.849.847.042.9

3. Length weighting alone does not produce the speedup. RFT (W1) uses the same length weighting. It keeps the base model's answer length through step 2000 and shortens only later. At a similar length (RFT step 3000: 130 tokens; TSD step 2000: 136 tokens), TSD's TPF is still twice as high (3.97 vs. 1.96; App. C.10).

What length weighting does. Combined with TSD, it keeps answers concise, which removes blocks to decode and matters most on large canvases. On MBPP at L=512, raw-reward TSD writes 237-token answers against 130 with weighting, at no higher cost per block (7.0 vs. 8.2). This explains the gap in Fig. 6/18. On GSM8K, the two effects contribute about equally to TSD's 3.7× NFE reduction: 1.7× from shorter answers (234 → 136 tokens) and 2.1× from more tokens per pass (1.85 → 3.97).

W3a (peak degradation)

TSD targets the low-NFE regime and does not always keep the peak at large budgets, mainly on MATH-500 and LiveCodeBench. On MBPP, GSM8K and Sudoku it matches or exceeds the base peak. As a LoRA adapter, it can be turned off when the budget is large. We discuss the confidence of TSD's early commitments in our response to the next point.

W3b (training stability)

We agree this needs clarification. The degradation depends on how large the update steps are and how many are taken; it is not built into TSD. K=6 and long training at our default learning rate both move the model faster and further from the base model. To test this, we retrained TSD on GSM8K with a 3× smaller learning rate (1e-6 vs. 3e-6), keeping everything else the same:

Learning rateStepAcc. (λ=0.99\lambda=0.99)NFE (λ=0.99\lambda=0.99)Acc. (λ=0.9\lambda=0.9)NFE (λ=0.9\lambda=0.9)
LLaDA (Fast-dLLM)80.7126.280.382.1
1×10−61\times10^{-6}100081.7114.581.473.6
1×10−61\times10^{-6}200081.284.979.452.5
1×10−61\times10^{-6}300081.563.178.737.4
1×10−61\times10^{-6}3736 (end)80.854.076.931.8
3×10−63\times10^{-6} (paper)100081.458.679.234.8
3×10−63\times10^{-6} (paper)200079.834.373.221.5
3×10−63\times10^{-6} (paper)300067.425.064.317.0

At the smaller learning rate, accuracy at λ=0.99\lambda=0.99 stays at base level for the whole run (3736 steps) while NFE falls from 126 to 54. The default rate reaches a similar point in about a quarter of the steps (step 1000: 81.4% at 58.6 NFE) and degrades only when training continues well past it. The learning rate and step count therefore set how far the model moves. In the paper, we use a fixed step budget for all tasks (2000 steps; 1000 for MATH-500). This experiment is added in App. C.9.

W3c (overconfidence)

We measured this directly by feeding the base model, RFT, dParallel and TSD identical partially masked GSM8K solutions (full table in our reply to Reviewer upfo and App. C.11). The reviewer is right that TSD sharpens its predictions: its top-1 accuracy is on par with the base model's (0.4–2.2 points higher), while its confidence is higher, so its ECE is also higher. Most of the extra tokens it commits at λ=0.99\lambda=0.99 are nevertheless correct: when half of the block is masked, TSD commits 79.8% of the masked tokens correctly against 69.5% for the base model, and its wrong commits stay at most 1.5% of the masked tokens when up to 75% of the block is masked (2.7% at 90%, against 0.2–0.3% for the base model). The sharpening is consistent with TSD losing more accuracy at a lower threshold (73.2% at λ=0.9\lambda=0.9 vs. 79.8% at λ=0.99\lambda=0.99). dParallel shows a similar degree of sharpening (ECE 17.7 vs. 17.9 at 90% masking), and RFT lies between the base model and TSD.

Q3 (non-verifiable domains)

The verifier only weights trajectories. It is not part of the distillation target. It is needed because masked diffusion with unmasking-only decoding never revises a committed token. In a wrong trajectory, the commitment-time teacher carries the error, and distilling toward it would make early predictions more confident about that error. The verifier removes these trajectories. Our no-reward ablation (App. C.5) shows this effect: without a verifier TSD still reduces NFE, but accuracy is not always preserved and the NFE reduction is smaller. For open-ended generation, any trajectory-level quality signal can replace the verifier, such as a reward model or an LLM judge. Samplers that can revise committed tokens, through remasking or uniform-state diffusion, should also make TSD less dependent on filtering. We have not yet evaluated open-ended generation and will discuss this in the Limitations.

We hope the RFT control, the separation of length from parallelism and the learning-rate experiment address the reviewer’s concerns, and we would be grateful if they would consider updating their score.

Reviewer_oxbvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces the Temporal Self-Distillation (TSD) approach to discrete diffusion language models (dLLMs), by developing an on-policy method that distills the model's denoising distribution from earlier time steps toward its distribution at the final time step (when a token is officially committed). This teaches the model at early stages to better anticipate its eventual output, allowing for much faster parallel decoding.

Critical strengths

A solid paper with sufficient ablation studies and an easy-to-follow algorithm to address an essential question in discrete diffusion language models (dLLMs): how to improve inference efficiency while maintaining generation quality, with relatively straightforward implementation. In contrast to traditional distillation methods that require massive, computationally expensive offline dataset generation from a separate "teacher" model, TSD generates its own teacher signals on the fly, hence avoiding a complex two-stage pipeline. Across seven distinct benchmarks covering mathematics, code, and planning, TSD substantially pushes the speed-quality frontier forward, performing exceptionally well in low-compute regimes.

Critical weaknesses

While TSD effectively accelerates dLLMs, discrete diffusion models as a whole still face general structural hurdles when competing against highly optimized autoregressive (AR) transformers on traditional causal text generation. Moreover, since the model learns from its own predictions at the final time step, the quality of distillation is constrained by the capability of the underlying dLLM itself.

Questions for authors
  1. Countdown results (Lines 321–338, Figure 4): The TSD curve shows substantially different accuracies at nearly identical NFEs. The authors should explain why different confidence thresholds lead to such large performance discrepancies despite similar inference costs.
  2. Inference time (Lines 270–274, 300–306): What is the actual wall-clock time per NFE across different methods? Although NFE provides a hardware-independent measure of inference cost, reporting actual inference latency would make the speedup claims more convincing.
  3. Ablation on β (Lines 151–161, 442–448): Rather than only comparing JS and KL divergence, providing an ablation study on the divergence parameter β would be a better way to understand its influence on training stability and inference performance.
  4. Ablation on γ_len (Lines 209–234, 432–440,Appendix C.6): The authors compare γ_len = 0and γ_len = 1, but do not examine intermediate values. A sensitivity analysis of γ_len would help clarify how the strength of length-aware reward weighting affects the trade-off between inference efficiency and generation accuracy.
  5. Reward consistency (Lines 186–190, 955–971, 1136–1143): Since the verifier V(•) and evaluation protocols differ in certain respects from the original methods, the authors should more clearly summarize these differences in the main text, in addition to presenting the exact structures primarily in the Appendix. In particular, they should clarify how differences in reward definitions, decoding procedures, and evaluation metrics affect the comparability between previous works and their procedures.
  6. Theoretical generalization (Lines 143–161, 407–431): Can the authors provide theoretical analyses or guarantees concerning the generalization of the proposed algorithm, particularly across different diffusion model families under mild assumptions. Such analysis would further strengthen the methodological contribution.
  7. Timestep-wise contribution (Lines 193–207, 1453–1485): An optional yet helpful theoretical analysis would be to investigate how different denoising timesteps contribute to the self-distillation loss based on current available data. Do different timesteps have roughly equal effects on model updates, or are the improvements concentrated in earlier or later denoising stages? This could provide additional insight into the mechanism of general self-distillation.
2 comments

We thank the reviewer for their positive assessment, in particular of the ablations and the single-stage on-policy design that avoids offline teacher generation. We address each question below. Changes in the revised PDF are marked in red.

W1 (comparison with AR models)

We agree that dLLMs still face open challenges relative to AR models. TSD addresses a complementary question: making parallel decoding, the main promised advantage of dLLMs, usable in practice.

W2 (bounded by the base model)

We agree, as stated in our Limitations. TSD is complementary to methods that add capability. Applied after RL (GDSD, ESPO), it keeps the RL gains while reducing NFE, and ESPO-TSD exceeds ESPO's reported numbers on HumanEval and MBPP. Jointly improving capability and speed in a single stage is a promising direction for future work.

Q1 (Countdown: different accuracy at similar NFE)

We logged the pass that commits each token (TSD, L=256L=256, all 256 problems) and split passes into those writing the answer and those filling positions after it (this is possible due to block diffusion).

λ\lambdaAcc.NFEAnswer passesMixed passesAfter-answer passes
0.535.5%20.18.81.310.1
0.9980.1%20.311.71.17.5

Paper checkpoint (β=0.5\beta=0.5), separate run with per-pass logging. The β\beta table below re-evaluates the same checkpoint at 79.3%.

TSD is confident on almost every token, so most tokens are committed in the first pass of their block at any λ\lambda. A low λ\lambda saves about 3 passes on the answer (8.8 vs. 11.7) but spends about as many after it (10.1 vs. 7.5), where it commits stray tokens instead of end-of-text tokens and needs extra passes around them. Total NFE stays near 20.

The accuracy gap comes from the numbers in the final expression, which must be consistent with each other. At λ=0.99\lambda=0.99 they are committed in separate passes, each conditioned on the previous ones. At λ=0.5\lambda=0.5 several are committed in the same pass. With 49, 53, 55 and target 51, the reasoning plans 55, 53, 49 at both thresholds, but λ=0.5\lambda=0.5 writes 53−53+4953 - 53 + 49. Repeated numbers appear in 12.5% of expressions at λ=0.5\lambda=0.5 and 1.6% at λ=0.99\lambda=0.99. A high threshold costs almost nothing here.

Q2 (wall-clock time per NFE)

We measured this on a single A100-40GB GPU with batch size 1 (full table in our reply to Reviewer upfo and App. C.12). One forward pass takes about 46 ms for both the base model and dParallel on GSM8K (50 ms on HumanEval), and 60 ms for TSD (66 ms on HumanEval). TSD is slower per pass only because we keep its LoRA adapter as a separate module: merging the small update into the bf16 weights rounds part of it away and changes the outputs. This overhead comes from the implementation, not from TSD. Because TSD needs far fewer passes, it still has the lowest end-to-end latency at every threshold: 2.6–3.0× faster than the base model on GSM8K and 2.3–2.5× on HumanEval, against 1.9–2.3× and 1.6–1.9× for dParallel.

Q3 (ablation on β\beta)

Following the reviewer's suggestion, we trained TSD on Countdown with JSD β∈{0.1,0.9}\beta \in \{0.1, 0.9\}, using the paper's configuration otherwise; β=0.5\beta=0.5 is the paper checkpoint. As in the paper, all Countdown models are trained at canvas length L=128L=128, and we evaluate them at both L=128L=128 and L=256L=256. Results at λ=0.99\lambda=0.99 (accuracy % / NFE):

ModelL=128L=128L=256L=256
GDSD, no TSD75.4 / 92.183.6 / 102.4
TSD, β=0.1\beta=0.179.7 / 12.681.6 / 18.1
TSD, β=0.5\beta=0.563.3 / 17.079.3 / 20.3
TSD, β=0.9\beta=0.971.5 / 16.076.2 / 21.3

Training is stable for every β\beta: the reward and NFE curves nearly coincide. All variants reduce NFE by 5 to 7×, and their answers have the same length (112 tokens at L=256L=256). β\beta mainly affects accuracy, and β=0.1\beta=0.1 is best. The differences are largest at L=128L=128, where many outputs reach the end of the canvas before writing the final expression; this happens less often with β=0.1\beta=0.1.

Q4 (ablation on γlen\gamma_{\text{len}})

We trained two more coding models with γlen∈\gamma_{\text{len}} \in {0.25, 0.5} (γlen=0\gamma_{\text{len}}=0: raw reward; larger values penalize long answers more), using the paper's configuration otherwise, and compare them with γlen=0\gamma_{\text{len}}=0 (Fig. 6) and the paper's γlen=1\gamma_{\text{len}}=1. Results at L=256L=256 (accuracy / NFE / answer length):

ModelMBPP, λ\lambda=0.9MBPP, λ\lambda=0.99HumanEval, λ\lambda=0.9HumanEval, λ\lambda=0.99
LLaDA (Fast-dLLM)44.0 / 83.2 / 21544.7 / 119.2 / 216––
TSD, γ_len=038.5 / 37.0 / 22350.6 / 57.7 / 21625.0 / 38.4 / 23232.9 / 62.8 / 230
TSD, γ_len=0.2538.5 / 35.6 / 21946.7 / 58.0 / 21329.9 / 37.9 / 22840.9 / 64.4 / 232
TSD, γ_len=0.540.1 / 29.5 / 19448.2 / 44.2 / 18823.8 / 32.5 / 20233.5 / 52.3 / 204
TSD, γ_len=1 (paper)46.3 / 25.5 / 12448.2 / 39.3 / 12536.6 / 28.1 / 14236.6 / 45.8 / 140
  • γ_len controls answer length: from 223 to 124 tokens on MBPP and from 232 to 142 on HumanEval as γ_len goes from 0 to 1.
  • At λ\lambda=0.9, γ_len=1 has the highest accuracy and the lowest NFE on both MBPP (46.3% at 25.5 NFE) and HumanEval (36.6% at 28.1 NFE).
  • At λ\lambda=0.99, γ_len=1 still has the lowest NFE, while weaker penalties reach a higher peak: 50.6% for γ_len=0 on MBPP (57.7 NFE) and 40.9% for γ_len=0.25 on HumanEval (64.4 NFE).
  • Without length weighting, TSD still halves NFE: at γ_len=0 its MBPP answers are as long as the base model's (216 tokens), yet it needs 57.7 instead of 119.2 forward passes, at higher accuracy (50.6% vs. 44.7%).

Q5 (reward consistency)

We summarize the differences below; the table is added into Section 4.1:

OursOriginal works (d1, GDSD, ESPO)Effect
Training rewardCorrectness only. Code: test pass rate + formatCorrectness + format rewards (GRPO-style)Only changes how TSD weights its own rollouts. Not used in evaluation
DecodingFast-dLLM for every model, baselines includedLow-confidence fixed-step decodingAbsolute numbers can differ from the original papers. We also show ESPO under its original decoding (Fig. 5, 13 to 15)
MetricsExact match, symbolic equivalence, rule-based checker, per-cell accuracy, pass@1Same benchmarksIdentical for every model we report
ProtocolSame prompts, problems, seed and threshold sweep for all modelsVariesAll our comparisons are matched

In short, the reward definition affects only TSD's training signal. The main deviation from the original papers is the decoder, which we change for all models alike so that NFE is comparable.

Q6 (theoretical generalization)

We do not provide formal guarantees. The commitment-time target is well defined for any masked dLLM decoded by unmasking only, since a committed token never changes afterwards. Uniform-state diffusion resamples every position, so there we use the settle step, the last step at which a token changes, as a heuristic target (Sec. 4.3) and validate it on Duo. As stated in our Limitations, this heuristic is validated only on a small Duo model.

Q7 (timestep-wise contribution)

In our new runs we logged, for every training sample, the sampled step t^\hat t, the number of decoding steps TcT_c of its trajectory and the loss, and binned the loss by the relative position t^/Tc\hat t/T_c (0 = start of decoding, 1 = end).

  • Coding: the mean loss is nearly flat across positions (0.21, falling to 0.17 in the last tenth) and stable over training, so early and late steps contribute about equally.
  • Countdown: in the first 500 training steps, the earliest decoding steps dominate (29% of the loss comes from t^/Tc<0.1\hat t/T_c<0.1). After 1500 steps, the loss shifts to the middle of the trajectory. By then each block is decoded in about three passes, so at most sampled steps no masked position is left to supervise and the training signal shrinks.

Gradient norms are logged once per optimizer step, so they cannot be split by decoding step; their correlation with the mean t^/Tc\hat t/T_c of a step is weak (∣ρ∣≤0.16|\rho| \le 0.16).

We hope these additions answer the reviewer’s questions and further strengthen the paper.

Reviewer_upfoOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose Temporal Self-Distillation (TSD) that aims to make diffusion language model decode reliably with far fewer forward passes. TSD trains early-step masked prediction to match the model’s own prediction at the step where that token is committed.

Critical strengths
  • The objective is one line and cheap. Teacher logits are already computed during the rollout and training needs one extra forward/backward per trajectory.
  • The results show significant improvements at low inference budgets.
  • Results on GDSD and ESPO suggest that TSD can accelerate models after RL training.
  • TSD is compatible with uniform-state diffusion as well as masked diffusion.
Critical weaknesses
  • The distinctive contribution needs stronger experimental isolation. Appendix A identifies closely related methods, including COPSD and dOPSD, but provides no direct comparison.
  • The gains are confounded with reward-weighted in-domain training. TSD is trained on each task’s training split with verifier rewards and on the exact prompt scaffold used at evaluation. Base LLaDA has no task-specific training, and dParallel and SDTT train without rewards.
  • Lower NFE could partly result from shorter outputs and sharper but not more accurate early predictions, alongside improved early predictions. Although the reward and length ablations help, measurements of token accuracy and calibration at matched partial contexts, together with output-length distributions and tokens per forward pass, would better establish the claimed mechanism.
Questions for authors
  • Could the authors provide comparisons to more closely related baselines such as COPSD and dOPSD?
  • Could the authors also provide inference latency or throughput measurements?
4 comments

We thank the reviewer for noting the simplicity of the objective, the significant gains at low inference budgets and that TSD applies after RL post-training and to uniform-state diffusion. We address each point below. Changes in the revised PDF are marked in red.

W1 & Q1 (comparison with COPSD / dOPSD)

We thank the reviewer for pointing us to these closely related works. Both appeared in July 2026, within the window that ICLR's guidelines treat as contemporaneous with our submission. A faithful comparison is also not possible yet. COPSD has released neither code nor checkpoints and omits key training details (learning rate, step count, fine-tuning strategy). dOPSD releases no checkpoints and reports results only with one-token-per-step decoding, not Fast-dLLM. We have sharpened the structural comparison in Appendix A:

  • dOPSD averages the model's predictions over all future steps at which a position is still masked. Under unmasking-only decoding, TSD's target is the last of these, the prediction at the step where the token is committed. It costs one teacher forward pass instead of one per remaining step. When the model's predictions are consistent across steps, the two targets coincide. dOPSD also targets reasoning quality, while TSD targets fewer decoding steps.
  • COPSD pulls early predictions toward an unnormalized target built from later decoding states and shared across a decoding stage. TSD instead matches a normalized distribution at each position's own commitment step.

W2 (confound with reward-weighted in-domain training)

To isolate whether the speedup comes from our algorithm or from these training factors, we trained a rejection-sampling fine-tuning (RFT) control with exactly TSD's training setup (data, prompts, rewards, length weighting, rollouts, hyperparameters). The only difference is the loss: RFT uses the standard masked-diffusion loss on the weighted rollouts. Results on the GSM8K test set (L=256, Fast-dLLM, all 1319 problems):

λ\lambdaModelAcc. (%)NFE ↓Answer lengthTokens/NFE ↑
0.99LLaDA80.7126.22341.85
0.99RFT, step 100076.9114.22201.92
0.99RFT, step 200072.3103.12242.17
0.99TSD, step 200079.834.31363.97
0.9LLaDA80.382.12342.85
0.9RFT, step 200073.562.42243.60
0.9TSD, step 200073.221.51376.38
  • RFT barely accelerates decoding. At either checkpoint, RFT reduces NFE by only 1.1–1.2× and loses accuracy. TSD reduces NFE by 3.7× at near-base accuracy (79.8% vs. 80.7%). TSD at λ=0.99\lambda=0.99 is both more accurate than RFT at λ=0.9\lambda=0.9 and needs 1.8× fewer NFE.
  • Length weighting alone does not shorten answers. RFT's answers stay almost as long as the base model's (224 vs. 234 tokens), and TSD commits 1.8× more tokens per forward pass than RFT (3.97 vs. 2.17). We separate length from parallelism in W3.
  • Rewards do not cause the speedup. TSD still reduces NFE without a verifier (App. C.5, Fig. 17), where rewards mainly preserve accuracy. On Countdown, GDSD and ESPO were already RL-trained with verifier rewards on the same task, yet applying TSD to them still reduces NFE by up to 4.5× (App. C.2, Figs. 11 and 15).

We therefore attribute the speedup to the temporal self-distillation objective rather than to reward-weighted in-domain training. We have added it in App. C.10.

W3 (shorter outputs vs. sharper predictions; calibration at matched contexts)

Output length and tokens per forward pass. These are reported in the controlled RFT table above (GSM8K, L=256). The answer-length distribution (p10 / median / p90, λ=0.99\lambda=0.99) is 178 / 254 / 254 tokens for LLaDA, 128 / 254 / 254 for RFT and 86 / 129 / 198 for TSD. TSD's answers are shorter, but shorter answers alone do not explain the speedup. Shortening reduces the number of blocks to decode, but not the number of tokens committed per forward pass, and TSD commits 2.1× more tokens per pass than LLaDA (3.97 vs. 1.85) and 1.8× more than RFT (3.97 vs. 2.17). Counting only the forward passes spent on blocks that contain answer tokens, which removes the effect of answer length, TSD commits 4.39 tokens per pass, against 1.86 for LLaDA and 2.19 for RFT.

Token accuracy and calibration at matched partial contexts. We fed the base model, the RFT control, dParallel and TSD (GSM8K, step 2000) identical partially masked inputs and compared their predictions on the masked positions. Each input is a GSM8K reference solution in the evaluation answer format. To mimic block decoding, earlier blocks are fully visible, the current 32-token block is masked at ratio rr, and later blocks are fully masked. We use the 1311 test problems whose formatted reference fits the 256-token canvas and 3 mask seeds (results vary by at most 0.1 points across seeds), and score the masked answer tokens of the current block. Accuracy is top-1 accuracy, Confidence the mean maximum probability, and ECE the expected calibration error (15 bins). Fast-dLLM commits a token in the current pass when its confidence exceeds λ=0.99\lambda=0.99; correct / wrong commits are the fractions of masked tokens committed this way that are predicted correctly / incorrectly.

rrModelAccuracyConfidenceECECorrect commits ↑Wrong commits ↓
50%LLaDA92.494.01.669.50.3
RFT92.294.72.572.50.5
dParallel92.796.94.281.61.2
TSD92.896.23.579.80.9
75%LLaDA83.085.52.649.30.3
RFT82.586.84.453.10.5
dParallel83.592.18.664.01.6
TSD84.191.67.665.51.5
90%LLaDA61.166.65.724.90.2
RFT60.569.28.829.40.5
dParallel61.679.217.735.21.8
TSD63.381.217.942.72.7

TSD trains the model's early predictions to match the ones it commits later. The table measures exactly these early predictions. At every tested mask ratio, TSD's early top-1 accuracy exceeds the base model's by 0.4–2.2 points. What changes is their confidence: many more of them become confident enough to be committed in the current pass, and almost all of them are correct, although confidence rises somewhat more than accuracy (higher ECE, as for dParallel). When half of the block is masked, TSD commits 79.8% of the masked tokens correctly against 69.5% for LLaDA; when 90% is masked, 42.7% against 24.9%, with at most 2.7% wrong commits. Committing more correct tokens in each pass is what reduces the number of passes, independently of answer length.

Q2 (latency / throughput)

We thank the reviewer for this suggestion. We measured inference latency on a single NVIDIA A100-40GB with batch size 1, using the same threshold-based parallel decoding for all models (L=256, block length 32), on 200 random GSM8K test problems and all 164 HumanEval problems. The models are LLaDA-8B-Instruct, dParallel-LLaDA-8B-Instruct and TSD (step 2000). Each cell shows mean latency per answer (s) / tokens per forward pass (TPF).

TaskMethodλ=0.8\lambda=0.8λ=0.9\lambda=0.9λ=0.95\lambda=0.95λ=0.99\lambda=0.99
GSM8KLLaDA2.94 / 3.623.73 / 2.844.41 / 2.405.73 / 1.84
dParallel1.32 / 5.781.62 / 4.781.99 / 3.962.94 / 2.67
TSD1.11 / 7.231.28 / 6.261.49 / 5.372.04 / 3.88
HumanEvalLLaDA3.69 / 3.054.53 / 2.465.25 / 2.126.65 / 1.67
dParallel1.91 / 4.442.40 / 3.552.90 / 2.964.05 / 2.14
TSD1.48 / 6.161.79 / 5.052.10 / 4.392.85 / 3.19

TSD has the lowest latency and the highest TPF at every threshold: it is 2.6–3.0× faster than LLaDA on GSM8K and 2.3–2.5× on HumanEval, against 1.9–2.3× and 1.6–1.9× for dParallel. These numbers are conservative for TSD. We keep its LoRA adapter as a separate module, because merging it into the bf16 weights rounds away the small update, so each TSD forward pass is about 30% slower than the base model's (60 vs. 46 ms per forward pass on GSM8K). We have added these results in App. C.12.

We hope the RFT control, the matched-context analysis and the latency measurements address the reviewer’s concerns, and we would be grateful if they would consider updating their score.

Thank you for your response. My concerns have been fully addressed. Therefore, I'll raise my score to a 3.

We sincerely thank the reviewer for the careful reading of our response and for raising their score. We sincerely appreciate your constructive feedback and thoughtful evaluation, which has directly contributed to strengthening the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.