- Rating
- 1(−1)
- Confidence
- 4(+1)
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes dSpark, a framework for accelerating diffusion language models through offline schedule construction and subsequent fine-tuning. Given a correct reference response, a frozen base model constructs a reveal schedule by promoting plausible reference tokens whose revelation reduces the surprisal of other masked reference tokens. The student is then trained on intermediate states from these schedules using masked-token cross-entropy and confidence sharpening, together with selective KL regularization on failed trajectories. At inference, the fine-tuned model uses confidence-based parallel decoding without the offline search. Experiments on LLaDA and Dream across mathematical reasoning and code generation benchmarks report improved accuracy–parallelism trade-offs, with ablations examining the promotion strategy and regularization.
Critical strengths
The paper provides empirical evidence that its training recipe improves decoding efficiency. In particular, Table 11 compares the proposed promotion criterion with natural scheduling, random plausible promotion, and highest-surprisal promotion. With KL regularization, the proposed criterion achieves 10.1 tokens per forward at 78.9% accuracy on LLaDA GSM8K, compared with 7.5 tokens per forward at 78.7% accuracy for random plausible promotion. This supports the practical value of the proposed training-state construction beyond simply promoting arbitrary plausible tokens. I regard this controlled empirical evidence as the paper’s main strength.
Critical weaknesses
-
The connection between finding a better reveal order and learning a better commitment strategy is insufficiently established. Section 4.2 constructs schedules for a fixed reference response using a frozen teacher. However, Section 4.3 trains the student to predict all eligible masked reference tokens from the resulting intermediate states, with additional confidence sharpening. The objective does not explicitly supervise which positions should be committed next, and I did not find a direct analysis showing that the student learns the useful commitment behavior identified by the offline search. Table 11 demonstrates that the proposed training-state construction improves final performance, but it does not establish whether this improvement comes from better commitment decisions, more effective supervised contexts, or changes in confidence. Furthermore, fine-tuning changes the model’s conditional predictions, and its generated responses may also change, leaving the relevance of the teacher’s optimized schedules to the student’s actual generation insufficiently explained. Exact reproduction of the original order is unnecessary, but the paper needs clearer evidence connecting its scheduling principle to the student’s inference behavior. This gap weakens the justification of the central “schedule distillation” claim.
-
The conceptual contribution is limited relative to the number of components and the additional trajectory-construction effort. The method combines candidate filtering, greedy promotion search, training on selected trajectory states, confidence sharpening, and selective KL regularization. Although the individual components have understandable purposes, I find their combination insufficiently supported by a clear, unified explanation. The existing ablations establish the effectiveness of the assembled recipe more convincingly than they establish a distinct new insight. In particular, the paper does not sufficiently justify whether the additional computation spent constructing optimized trajectories teaches useful behavior that simpler training could not obtain at a comparable total cost. The offline construction cost is also insufficiently characterized. Overall, I view the contribution as a carefully assembled empirical recipe with limited conceptual advance, which substantially reduces its significance in my assessment.
Questions for authors
-
Clarification of the claimed learning mechanism. What evidence shows that training on the optimized trajectory states causes the student to make more useful commitment decisions during its own generation? In particular, do the student’s actual early commitments have greater downstream utility than those of students trained with natural or random promotion schedules? Please also clarify how the interpretation of “schedule distillation” accounts for changes in the student’s conditional predictions and generated responses after fine-tuning.
-
Clarification of the ablation controls. In Table 11, how are states sampled from trajectories of different lengths, and are the comparisons matched for training updates, sampled-state counts, and total supervised tokens? Is there an analysis separating the effect of optimized-state construction from confidence sharpening? These details would help determine whether the results support learning a better commitment strategy or a broader benefit from changing the distribution and weighting of supervised examples.
6 comments
Response to Reviewer t3Pk [2/4]
W1.2: Furthermore, fine-tuning changes the model’s conditional predictions, and its generated responses may also change, leaving the relevance of the teacher’s optimized schedules to the student’s actual generation insufficiently explained. Exact reproduction of the original order is unnecessary, but the paper needs clearer evidence connecting its scheduling principle to the student’s inference behavior. This gap weakens the justification of the central “schedule distillation” claim.
Q1.2: Please also clarify how the interpretation of “schedule distillation” accounts for changes in the student’s conditional predictions and generated responses after fine-tuning.
Thanks for raising this point. We clarify that the analysis in Table 1 was conducted in exactly this setting. The fine-tuned student decodes its own responses to unseen test problems without access to any schedule, and the offline schedule is computed on those same responses. If the teacher's schedules were no longer relevant after fine-tuning, dSpark should behave like the control student trained on natural schedules. Instead, its early commitments are more helpful to the remaining positions, and its decoding order moves about three times as far toward the offline schedule.
We believe the connection is preserved for two reasons. First, the LoRA adaptation and the KL term keep the student close to the teacher. Second, the usefulness of a promoted token is largely determined by the content of the response. For example, an intermediate result that later steps depend on stays useful to commit early, and fine-tuning is not expected to alter such dependencies. Schedule distillation therefore transfers the scheduling principle, which is to commit high-utility tokens early, and does not require the student to reproduce the teacher's exact order. The revised Section 5.3 now states this.
W2.1: The conceptual contribution is limited relative to the number of components and the additional trajectory-construction effort. The method combines candidate filtering, greedy promotion search, training on selected trajectory states, confidence sharpening, and selective KL regularization. Although the individual components have understandable purposes, I find their combination insufficiently supported by a clear, unified explanation. The existing ablations establish the effectiveness of the assembled recipe more convincingly than they establish a distinct new insight.
Thanks for the comments. We clarify that dSpark follows from a single observation. Decoding speed depends on the order in which tokens are revealed, and the most confident token is often not the most useful one to reveal (Fig. 1). We formalize this with the uncertainty area (Eq. 4), and each component serves this one objective. The promotion rule (Eq. 6) is a greedy step that reduces the area, and the candidate filter restricts it to tokens the model itself finds plausible. Training on the resulting states transfers the order to the student. Sharpening is needed because the decoder commits by confidence, so a learned order can only be expressed through confidence, and the KL term confines this confidence gain to correct predictions. The method therefore consists of one schedule and two loss terms on top of cross-entropy.
The ablations also isolate the insight from the recipe. In Table 11 of the submission (now Table 15), the data, losses and training budget are fixed and only the reveal order changes. TPF increases step by step from the natural schedule to random promotion and to highest-surprisal promotion, and it is highest with our criterion. The gain therefore comes from the utility principle itself, and not from early reveals in general or from the other components.
Response to Reviewer t3Pk [3/4]
W2.2: In particular, the paper does not sufficiently justify whether the additional computation spent constructing optimized trajectories teaches useful behavior that simpler training could not obtain at a comparable total cost. The offline construction cost is also insufficiently characterized. Overall, I view the contribution as a carefully assembled empirical recipe with limited conceptual advance, which substantially reduces its significance in my assessment.
Thanks for this comment. Following your suggestion, we have added a cost breakdown and an equal-cost comparison to Appendix A.7 of the revised paper (Table 11). On LLaDA, generating the responses takes 200 A100 GPU-hours, a cost shared by any method that distills from model trajectories. Constructing the optimized schedules adds 13 GPU-hours, about 6% of the total. It requires 20.4 teacher forward passes per full response on average, since all candidates of a round (4.3 on average) are scored in one batched pass. Training takes 10 GPU-hours for 6 epochs, and inference incurs no additional cost. To compare at equal cost, we gave the construction budget to the simpler alternative and trained the natural-schedule student for 14 epochs, which brings it to the same total cost as dSpark (Table 2).
Table 2. Equal-cost comparison on GSM8K with LLaDA. Both students spend 23 A100 GPU-hours after response generation.
| Student | Cost after response generation | TPF | Acc. (%) |
|---|---|---|---|
| Natural schedule, 14 epochs | 23 GPU-hours | 8.8 | 77.01 |
| dSpark, construction and 6 epochs | 23 GPU-hours | 10.12 | 78.94 |
At the same total cost, the natural-schedule student remains behind dSpark in both parallelism and accuracy. The optimized schedules therefore teach a behavior that additional training on natural schedules does not provide. This is consistent with Table 1, where dSpark moves much further toward the offline schedule than the natural-schedule student.
Q2.1: Clarification of the ablation controls. In Table 11, how are states sampled from trajectories of different lengths, and are the comparisons matched for training updates, sampled-state counts, and total supervised tokens?
Thanks for the question. Yes, the comparisons are matched. Training updates and sampled-state counts are identical across rows, and supervised tokens agree to within 0.1 token per state. All rows of Table 11 of the submission (now Table 15) use the same sampling procedure. Each response contributes exactly one state per epoch, regardless of the length of its trajectory. We sample a block uniformly and then a round uniformly within that block, and the state is redrawn at every epoch. Longer trajectories therefore receive neither more states nor more weight. All rows also use the same 52k correct responses (and, with KL, the same 13k failed responses), the same 6 epochs and the same batch size of 64, so the number of training updates and the number of sampled states are identical. The number of supervised tokens per state is determined by the schedule and in practice it is nearly the same. On average a state from a correct response supervises 49.2 tokens under the natural schedule and 49.3 under ours, and a state from a failed response supervises the same 127.8 tokens in every row. The loss is averaged over the supervised tokens of each batch, so every update carries the same weight in all rows.
Response to Reviewer t3Pk [1/4]
Thank you for your constructive and valuable comments, which are definitely helpful for us to improve the quality of the paper. We have uploaded a revised version of the manuscript to accommodate all your mentioned suggestions. Below, we further address each of your concerns as follows:
Thanks for the comment. We clarify that the decoder commits the tokens for which the model has the highest confidence, so the model's confidence under the current context determines each commitment decision. Our training states shape exactly this confidence. A promoted token is a prediction target only in the early states, before the schedule commits it, so the student learns to predict it confidently at that early point. To verify the effect directly, we compared dSpark with a control student trained with the same losses (sharpening included) but on natural schedules. Both students generate freely on the GSM8K test set, and the frozen base model evaluates every token they commit.
Table 1. Commitment of high-utility tokens during free generation on the GSM8K test set. Both students are trained with the same losses, sharpening included, and differ only in the training schedule. ↑ means higher is better.
Early commits are tokens that a student commits while the base model would still keep them masked. Utility measures how much these tokens help predict the other masked positions (Eq. 6), relative to tokens left masked, so 1$\times$ means no difference. Schedule gain recovered measures how far the decoding order moves from the base model's order (0%) toward the offline schedule (100%).
dSpark makes clearly better commitment decisions than the natural-schedule student. It commits more tokens early, the tokens it commits early are more useful to the remaining positions, and its decoding order moves about three times as far toward the offline schedule. The two students share the same losses, sharpening included, so this improvement comes from the optimized schedule and not from higher confidence. The student therefore learns the commitment behavior identified by the offline search and applies it during its own generation. This analysis is now in Section 5.3 of the revised paper (Table 4).