- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper converts strong AR language models into diffusion language models, first proving that existing objectives—masked-ELBO teacher forcing (SDAR) and on-policy token distillation (OPDLM)—fit intermediate predictions and can stabilize wrong output distributions. DM-OPD instead matches, at student-generated prefixes, the completed-block text distribution induced by the student’s decoder to the frozen teacher, using an auxiliary AR density estimate, a dependency-based credit decomposition preserving the expected gradient, and a stochastic keep-or-replace decoder with trainable action probabilities. Trained from Qwen3-1.7B/4B without answer-correctness rewards, the 1.7B student beats SDAR-Chat and OPDLM on all eight benchmarks at 4K/8K, and the 4B student is within ~1 point of its AR teacher at 8K (AIME25: 33.12 vs. 33.96). A matched ablation credits the objective itself with +5.44 points over OPDLM’s, whose loss falls ~9× while accuracy stays below initialization.
Critical strengths
- A formal, empirically corroborated dissection of the objective mismatch in dLLM distillation. Proposition 1 and the fixed-point analyses (A.2, A.3) show that intermediate-prediction losses can stabilize wrong output laws even on student trajectories, and run B reproduces this at scale: OPDLM’s loss falls ~9× while accuracy stays below initialization.
- A principled pipeline with unusually careful validation. The auxiliary density estimate, credit decomposition, and keep-or-replace decoder make the completed-text KL trainable, with guarantees honestly delimited (Prop. 3(iii)). The evaluation spans eight benchmarks, two sizes, both thinking modes, frozen graders, and controlled ablations isolating the objective (+5.44) and the density term (−5.64); the 4B student is within ~1 point of the AR teacher on four benchmarks at 8K, and favorable-to-baseline cases are reported honestly.
Critical weaknesses
- The efficiency story relative to the AR teacher is not yet closed. At the best-quality point (B4I4), TPF is 1.0 and measured latency is ~2× AR (Table 13); faster I3/I2 decoding trades against marked quality drops (e.g., MATH500 73.35→41.05). Gains are therefore clearest relative to other diffusion conversions, not the AR teacher at equal quality, so the “why not keep the AR model?” question remains only partially addressed.
- Attribution and comparability caveats in the headline comparisons. (i) Table 1 contrasts thinking-trained students with OPDLM checkpoints trained without thinking, partially mitigated by the non-thinking run and OPDLM-MATH-Thinking. (ii) Matched-update ablations still consume ~150M auxiliary tokens, and the monitoring panel is drawn from the reported benchmarks. (iii) The OPDLM-MATH reproduction (30.63 vs. its reported 43.3) is unreconciled; if realizable, the specialist result could flip. Together these leave the headline attribution robust but not fully isolated.
Questions for authors
- About the Budget-matched attribution control, can you report a control granting the OPDLM-objective arm the same total rollout-token budget used by the complete DM-OPD pipeline? This would clarify whether the Appendix G gain reflects the completed-text objective rather than the auxiliary stream’s extra generated tokens.
- Considering teacher-parity efficiency, is there any operating point where DM-OPD matches the AR teacher’s quality while exceeding its measured throughput or reducing wall-clock latency? If not, how should the deployment advantage over simply serving the AR teacher be stated?
2 comments
Response to Reviewer P5Qe (2/2)
2. Comparability, Evaluation Audit, and OPDLM-MATH Budget Resolution
Our revised comparisons match each model's training and evaluation modes (Table 1), include three benchmarks that are completely unmonitored during training (Appendix B.5), and resolve the specialist OPDLM-MATH-4B-Thinking gap at a 32K generation budget where OPDLM-MATH approaches its published scores (42.08% AIME24, 90.60% MATH500) while generalist DM-OPD reaches 71.25% and 96.40% (Tables 1, 18).
-
Specialist generation budget (Table 18; Appendix G.1): Under the shared 8K protocol, OPDLM-MATH scores 30.63%, 25.00%, and 85.90% on AIME24, AIME25, and MATH500 because over 80% of its AIME responses truncate at the 8K cap. At 32K it reaches 42.08%, 44.17%, and 90.60%, matching its published AIME24/MATH500 results. On the identical 32K prompt panel, our generalist 4B student reaches 71.25%, 64.58%, and 96.40%, resolving the discrepancy without inferring an unreported budget. The ±7.23 in Table 18 is a question-clustered standard error reflecting variation across problems. On the fixed 30 AIME24 problems, the estimated sampling standard error of our eight-draw reproduction mean is 1.96 percentage points (Appendix G.1).
-
Monitoring and prompt overlap (Appendices B.5, E.1): Yes—the reduced 2,977-response training monitor reuses original questions from the five mathematics/science benchmarks. The original 4B checkpoint (step 137) stopped at a wall-clock cutoff while still improving, so we continued training under the same settings to a fixed 400-update budget, set before its endpoint evaluation. Some accelerated runs stop on monitoring saturation (Table 8). Final evaluation uses the full panels without repetition cutoffs, and MMLU, IFEval, and HumanEval were never monitored (on these three unmonitored benchmarks, 4B non-thinking DM-OPD averages 75.37% at 4K vs. 59.83% and 58.93% for the controls; Table 19). The separate prompt-overlap audit finds four MATH500 matches and none in the other seven benchmarks; excluding those four shifts macro accuracy by at most 0.0524 points (Appendices B.5, E.1).
3. Quality–Speed Frontier and Deployment Positioning vs. AR
B4I3 is our demonstrated lower-latency operating point relative to AR. At 4K in matched thinking evaluation, it retains near-teacher mathematical reasoning (92.72% GSM8K and 75.15% MATH500 vs. 93.50% and 77.80% for AR; Table 2) while cutting fixed-4K BS1 latency from 15.564 s to 13.548 s (1.15x speedup; Table 29). The original latency figures (Table 13 of the submission) were inflated by performance bugs in our serving engine that are now fixed; the figures here are re-measured on the fixed engine. Faster schedules retain a trade-off across the full eight-benchmark panel, while B4I4 remains the quality-focused configuration (0.86x AR speed at 4K; 70.90% 8K macro accuracy vs. 71.62% for AR; Table 1).
At nominal TPF 2, widening the block from B4I2 to B8I4 improves all five mathematics/science benchmarks—lifting MATH500 from 69.85% to 72.80% and GSM8K from 84.14% to 87.19% (Table 2)—while retaining a 1.67x BS1 speedup at 4K (Table 30). Tables 29–30 report the full measured trade-off, supporting parallel block refinement when single-request latency is prioritized and exposing remaining full-panel gaps.
Because these accelerated schedules use a fixed number of passes per block, integrating adaptive pass allocation into our decoder-matched training is a complementary next step for concentrating refinement compute on harder blocks.
Response to Reviewer P5Qe (1/2)
We thank Reviewer P5Qe for the thoughtful review and specific questions on compute-matched attribution, evaluation comparability, and the deployment trade-off relative to AR decoding.
Updates during rebuttal. Compared with the original submission, the DM-OPD scores and figures reflect extended training and new 4B evaluations completed during rebuttal. The AR reference was re-evaluated on the same H200 hardware as the diffusion models to remove hardware differences, so its scores differ slightly from the original Table 1. The main tables and quality–speed charts now focus on 4B models, and the original 1.7B quality results remain in Appendix F. The original wall-clock table (1.7B, RTX 5090) is removed: those timings were inflated by several performance bugs in our serving engine, which are now fixed. The new 4B measurements use the fixed engine on an H200, with AR timed in the same engine (Appendix I). All citations below refer to the revised PDF, where key revisions are marked in blue; the original Appendices F, G, and H are now G, H, and I.
1. Matched-Compute 4B Objective Controls
At ~68 H200 GPU-hours per 4B non-thinking arm—counting all student rollouts, auxiliary rollouts, 4× auxiliary updates, and scoring, with student and auxiliary tokens accounted separately—DM-OPD leads extended-compute native OPDLM by +9.28 / +9.51 pp (4K/8K) and leads the matched-decoder OPDLM-loss control by +10.96 / +10.56 pp, even though the auxiliary-free control receives 2.08x more student steps and 2.55x more student response tokens (Tables 19, 21).
We use equal total method GPU-hours as the resource-controlled comparison because on-policy training expends compute across rollout generation, scoring, and optimization, including the auxiliary stream. Reporting the token streams separately (43.59M student + 173.60M auxiliary response tokens for DM-OPD versus 111.07M direct student tokens for the auxiliary-free control; Tables 6, 21) confirms the objective's advantage at equal measured training compute.