- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Temporal Self-Distillation (TSD), an on-policy method designed to accelerate inference in discrete diffusion language models (dLLMs), with a primary focus on masked diffusion models (MDMs). The key premise is that dLLMs suffer steep performance drops when decoding with low numbers of function evaluations (NFEs) because early-timestep predictions lack sufficient decoded context.
TSD addresses this by distilling the model's predictive distribution at an earlier, masked timestep toward its own predictive distribution at the commitment timestep . To make training computationally tractable, the authors introduce a single-timestep Monte Carlo approximation and weight the distillation loss using a verifier reward , augmented with a length-aware penalty to discourage verbosity. The method is evaluated across mathematical reasoning (GSM8K, MATH-500), planning (Countdown, Sudoku), and code generation (HumanEval, MBPP, LiveCodeBench) on top of LLaDA-8B and RL-post-trained checkpoints (GDSD, ESPO), with an extension to uniform-state diffusion
Critical strengths
- This paper has a clear motivation and addresses an important research question. Targeting the discrepancy between early-stage and commitment-stage token predictions is a natural and well-motivated avenue for enabling more aggressive parallel unmasking in dLLMs.
- Unlike offline distillation approaches (e.g., dParallel, SDTT) that require caching full teacher trajectories or maintaining separate teacher models, TSD derives its training targets dynamically along the model's own on-policy rollouts.
- The authors evaluate across diverse domains (code, math, planning), analyze varying canvas lengths and explore compatibility with RL post-training (GDSD, ESPO).
- The identification and discussion of KL divergence instability versus Jensen–Shannon Divergence (JSD) boundedness provides useful practical insight for distilling discrete distributions.
Critical weaknesses
On comparison with baselines
- A central methodological flaw in the main benchmark comparisons (Figures 3 and 4) is the baseline fairness. For GSM8K, MATH-500, and Coding (MBPP, HumanEval, LiveCodeBench), TSD fine-tunes LLaDA-8B-Instruct via LoRA on task-specific training sets: GSM8K training split, MATH training split (~7.5k), and KodCode-Light-RL-10k (Table 1). The primary baseline LLaDA (Fast-dLLM), is the off-the-shelf, general-purpose base model without any in-domain adaptation.
On decoding efficiency confounding factors
- In Fast-dLLM, decoding proceeds in fixed-size blocks (e.g., block length 32). In this setup, the total number of forward passes (NFE) depends strongly on the total sequence length: shorter generations require fewer total blocks to unmask, which mechanically lowers the NFE. The authors introduce Length-Aware Reward Weighting (Eq. 9), which explicitly penalizes completions exceeding a hand-picked target length . The ablation in Figure 6 and Figure 18 reveals that without length-aware weighting (raw-reward), the low-NFE advantage degrades considerably, particularly at larger canvas sizes. This brings a question of correctness: is TSD actually teaching the model to unmask more tokens in parallel at each step, or is it simply penalizing verbosity so that the model produces shorter answers that finish in fewer blocks?
On over confidence and peak degradation
- On MATH-500 (Figure 4), the base model achieves 35.4% accuracy at higher NFEs, whereas TSD tops out at 31.8%, a drop of 3.6 percentage points. Similarly, on LiveCodeBench (Figure 3), the base model surpasses TSD at higher NFEs.
- As shown in Appendix C.8 (Figures 20–22), increasing the number of Monte Carlo timesteps from K=1 to K=6 accelerates NFE reduction but causes accuracy to drop sharply after just 1,000 steps. The authors state on lines 278–282 that prolonged training overfits to verifier rollouts and erodes general reasoning.
- This indicates that TSD induces artificial probability sharpening (self-reinforcing overconfidence), which forces Fast-dLLM to prematurely commit tokens above threshold . If the model cannot safely sustain training beyond 1,000–2,000 steps without degrading, the method acts more as an early-stopping heuristic than a stable distillation procedure.
Questions for authors
- Controlled baseline
- Debiasing length from parallelism
- Generalization to non verifiable domains
2 comments
Part 2/2
W3a (peak degradation)
TSD targets the low-NFE regime and does not always keep the peak at large budgets, mainly on MATH-500 and LiveCodeBench. On MBPP, GSM8K and Sudoku it matches or exceeds the base peak. As a LoRA adapter, it can be turned off when the budget is large. We discuss the confidence of TSD's early commitments in our response to the next point.
W3b (training stability)
We agree this needs clarification. The degradation depends on how large the update steps are and how many are taken; it is not built into TSD. K=6 and long training at our default learning rate both move the model faster and further from the base model. To test this, we retrained TSD on GSM8K with a 3× smaller learning rate (1e-6 vs. 3e-6), keeping everything else the same:
| Learning rate | Step | Acc. () | NFE () | Acc. () | NFE () |
|---|---|---|---|---|---|
| LLaDA (Fast-dLLM) | 80.7 | 126.2 | 80.3 | 82.1 | |
| 1000 | 81.7 | 114.5 | 81.4 | 73.6 | |
| 2000 | 81.2 | 84.9 | 79.4 | 52.5 | |
| 3000 | 81.5 | 63.1 | 78.7 | 37.4 | |
| 3736 (end) | 80.8 | 54.0 | 76.9 | 31.8 | |
| (paper) | 1000 | 81.4 | 58.6 | 79.2 | 34.8 |
| (paper) | 2000 | 79.8 | 34.3 | 73.2 | 21.5 |
| (paper) | 3000 | 67.4 | 25.0 | 64.3 | 17.0 |
At the smaller learning rate, accuracy at stays at base level for the whole run (3736 steps) while NFE falls from 126 to 54. The default rate reaches a similar point in about a quarter of the steps (step 1000: 81.4% at 58.6 NFE) and degrades only when training continues well past it. The learning rate and step count therefore set how far the model moves. In the paper, we use a fixed step budget for all tasks (2000 steps; 1000 for MATH-500). This experiment is added in App. C.9.
W3c (overconfidence)
We measured this directly by feeding the base model, RFT, dParallel and TSD identical partially masked GSM8K solutions (full table in our reply to Reviewer upfo and App. C.11). The reviewer is right that TSD sharpens its predictions: its top-1 accuracy is on par with the base model's (0.4–2.2 points higher), while its confidence is higher, so its ECE is also higher. Most of the extra tokens it commits at are nevertheless correct: when half of the block is masked, TSD commits 79.8% of the masked tokens correctly against 69.5% for the base model, and its wrong commits stay at most 1.5% of the masked tokens when up to 75% of the block is masked (2.7% at 90%, against 0.2–0.3% for the base model). The sharpening is consistent with TSD losing more accuracy at a lower threshold (73.2% at vs. 79.8% at ). dParallel shows a similar degree of sharpening (ECE 17.7 vs. 17.9 at 90% masking), and RFT lies between the base model and TSD.
Q3 (non-verifiable domains)
The verifier only weights trajectories. It is not part of the distillation target. It is needed because masked diffusion with unmasking-only decoding never revises a committed token. In a wrong trajectory, the commitment-time teacher carries the error, and distilling toward it would make early predictions more confident about that error. The verifier removes these trajectories. Our no-reward ablation (App. C.5) shows this effect: without a verifier TSD still reduces NFE, but accuracy is not always preserved and the NFE reduction is smaller. For open-ended generation, any trajectory-level quality signal can replace the verifier, such as a reward model or an LLM judge. Samplers that can revise committed tokens, through remasking or uniform-state diffusion, should also make TSD less dependent on filtering. We have not yet evaluated open-ended generation and will discuss this in the Limitations.
We hope the RFT control, the separation of length from parallelism and the learning-rate experiment address the reviewer’s concerns, and we would be grateful if they would consider updating their score.
Part 1/2
We thank the reviewer for recognizing the clear motivation, the fact that the on-policy design avoids offline teacher trajectories and the breadth of the evaluation and ablations. We address each point below. Changes in the revised PDF are marked in red.
W1 & Q1 (baseline fairness, controlled baseline)
We agree that in-domain training must be controlled for. We trained a rejection-sampling fine-tuning (RFT) control with exactly TSD's training setup (GSM8K data, prompts, on-policy rollouts, verifier, length-aware weighting, hyperparameters). The only difference is the loss: RFT uses the standard masked-diffusion loss on the weighted rollouts. Results on the full GSM8K test set (L=256, Fast-dLLM):
This agrees with evidence already in the paper. On Countdown, GDSD was already RL-trained on the same task, yet applying TSD to it still reduces NFE by up to 4.5× (App. C.2). SDTT, trained on the same in-domain corpora, gives no NFE reduction (App. C.3). The full RFT results are in App. C.10; due to compute limits, this control was run on GSM8K only.
W2 & Q2 (shorter answers or more parallelism)
In Fast-dLLM, a shorter answer leaves fewer blocks with content, and blocks holding only end-of-text tokens are cheap but still cost at least one pass. We therefore report two measures that do not depend on length: tokens per forward pass (TPF = answer tokens / NFE) and NFE per answer block (passes spent on blocks containing answer tokens, divided by the number of such blocks). All results are at λ=0.99, L=256.
1. Without length weighting, TSD keeps the answer length but still decodes much faster.
On MBPP, raw-reward TSD (γlen=0) writes answers as long as the base model's (221 vs. 216 tokens), yet halves NFE and doubles TPF. On Countdown, its answers are only 9% shorter, but NFE drops 5.2×. Its cost per answer block is about the same as with length weighting (8.2 vs. 7.9 on MBPP).
2. At equal answer length, TSD needs about 2× fewer passes. Mean NFE on MBPP, grouped by answer length (tokens):
3. Length weighting alone does not produce the speedup. RFT (W1) uses the same length weighting. It keeps the base model's answer length through step 2000 and shortens only later. At a similar length (RFT step 3000: 130 tokens; TSD step 2000: 136 tokens), TSD's TPF is still twice as high (3.97 vs. 1.96; App. C.10).
What length weighting does. Combined with TSD, it keeps answers concise, which removes blocks to decode and matters most on large canvases. On MBPP at L=512, raw-reward TSD writes 237-token answers against 130 with weighting, at no higher cost per block (7.0 vs. 8.2). This explains the gap in Fig. 6/18. On GSM8K, the two effects contribute about equally to TSD's 3.7× NFE reduction: 1.7× from shorter answers (234 → 136 tokens) and 2.1× from more tokens per pass (1.85 → 3.97).