acceptodds
Under review as a conference paper at ICLR 2027

DM-OPD: Aligning Diffusion Language Models with Autoregressive Teachers

Abstract

Converting strong autoregressive (AR) models into diffusion language models promises parallel generation while retaining AR capabilities. Existing conversion methods fit intermediate predictions, but improving this fit need not improve the distribution of generated text. We analytically demonstrate this mismatch even on the student's own generation trajectories. We introduce Distribution Matching OPD (DM-OPD), which defines distillation on the text distribution induced by the student's actual decoder and derives an unbiased on-policy gradient estimator for each completed-block KL at student-generated prefixes. We make this objective trainable by estimating student text likelihoods, decomposing completed-text feedback into dependency-based action credit, and defining trainable probabilities for refinement decisions. Using Qwen3 initialization and teacher checkpoints with OPDLM's prompt corpus, we distill without answer-correctness rewards. Our 1.7B student outperforms the diffusion students SDAR and OPDLM on all eight benchmarks at 4K and 8K response limits, as does our 4B student except on HumanEval at 4K. With the same decoder and the guard disabled, our objective improves monitoring accuracy by 5.4 points over OPDLM's objective, whose loss keeps falling. Students trained for fewer refinement iterations outperform released diffusion models decoded with the same iteration budget on most benchmarks. These results establish completed-text distribution matching as an effective foundation for AR-to-diffusion distillation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Diffusion Language Models, On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_P5QeOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper converts strong AR language models into diffusion language models, first proving that existing objectives—masked-ELBO teacher forcing (SDAR) and on-policy token distillation (OPDLM)—fit intermediate predictions and can stabilize wrong output distributions. DM-OPD instead matches, at student-generated prefixes, the completed-block text distribution induced by the student’s decoder to the frozen teacher, using an auxiliary AR density estimate, a dependency-based credit decomposition preserving the expected gradient, and a stochastic keep-or-replace decoder with trainable action probabilities. Trained from Qwen3-1.7B/4B without answer-correctness rewards, the 1.7B student beats SDAR-Chat and OPDLM on all eight benchmarks at 4K/8K, and the 4B student is within ~1 point of its AR teacher at 8K (AIME25: 33.12 vs. 33.96). A matched ablation credits the objective itself with +5.44 points over OPDLM’s, whose loss falls ~9× while accuracy stays below initialization.

Critical strengths
  1. A formal, empirically corroborated dissection of the objective mismatch in dLLM distillation. Proposition 1 and the fixed-point analyses (A.2, A.3) show that intermediate-prediction losses can stabilize wrong output laws even on student trajectories, and run B reproduces this at scale: OPDLM’s loss falls ~9× while accuracy stays below initialization.
  2. A principled pipeline with unusually careful validation. The auxiliary density estimate, credit decomposition, and keep-or-replace decoder make the completed-text KL trainable, with guarantees honestly delimited (Prop. 3(iii)). The evaluation spans eight benchmarks, two sizes, both thinking modes, frozen graders, and controlled ablations isolating the objective (+5.44) and the density term (−5.64); the 4B student is within ~1 point of the AR teacher on four benchmarks at 8K, and favorable-to-baseline cases are reported honestly.
Critical weaknesses
  1. The efficiency story relative to the AR teacher is not yet closed. At the best-quality point (B4I4), TPF is 1.0 and measured latency is ~2× AR (Table 13); faster I3/I2 decoding trades against marked quality drops (e.g., MATH500 73.35→41.05). Gains are therefore clearest relative to other diffusion conversions, not the AR teacher at equal quality, so the “why not keep the AR model?” question remains only partially addressed.
  2. Attribution and comparability caveats in the headline comparisons. (i) Table 1 contrasts thinking-trained students with OPDLM checkpoints trained without thinking, partially mitigated by the non-thinking run and OPDLM-MATH-Thinking. (ii) Matched-update ablations still consume ~150M auxiliary tokens, and the monitoring panel is drawn from the reported benchmarks. (iii) The OPDLM-MATH reproduction (30.63 vs. its reported 43.3) is unreconciled; if realizable, the specialist result could flip. Together these leave the headline attribution robust but not fully isolated.
Questions for authors
  1. About the Budget-matched attribution control, can you report a control granting the OPDLM-objective arm the same total rollout-token budget used by the complete DM-OPD pipeline? This would clarify whether the Appendix G gain reflects the completed-text objective rather than the auxiliary stream’s extra generated tokens.
  2. Considering teacher-parity efficiency, is there any operating point where DM-OPD matches the AR teacher’s quality while exceeding its measured throughput or reducing wall-clock latency? If not, how should the deployment advantage over simply serving the AR teacher be stated?
2 comments
AuthorsOpenReview ↗

Response to Reviewer P5Qe (1/2)

We thank Reviewer P5Qe for the thoughtful review and specific questions on compute-matched attribution, evaluation comparability, and the deployment trade-off relative to AR decoding.

Updates during rebuttal. Compared with the original submission, the DM-OPD scores and figures reflect extended training and new 4B evaluations completed during rebuttal. The AR reference was re-evaluated on the same H200 hardware as the diffusion models to remove hardware differences, so its scores differ slightly from the original Table 1. The main tables and quality–speed charts now focus on 4B models, and the original 1.7B quality results remain in Appendix F. The original wall-clock table (1.7B, RTX 5090) is removed: those timings were inflated by several performance bugs in our serving engine, which are now fixed. The new 4B measurements use the fixed engine on an H200, with AR timed in the same engine (Appendix I). All citations below refer to the revised PDF, where key revisions are marked in blue; the original Appendices F, G, and H are now G, H, and I.

1. Matched-Compute 4B Objective Controls

At ~68 H200 GPU-hours per 4B non-thinking arm—counting all student rollouts, auxiliary rollouts, 4×4\times auxiliary updates, and scoring, with student and auxiliary tokens accounted separately—DM-OPD leads extended-compute native OPDLM by +9.28 / +9.51 pp (4K/8K) and leads the matched-decoder OPDLM-loss control by +10.96 / +10.56 pp, even though the auxiliary-free control receives 2.08x more student steps and 2.55x more student response tokens (Tables 19, 21).

We use equal total method GPU-hours as the resource-controlled comparison because on-policy training expends compute across rollout generation, scoring, and optimization, including the auxiliary stream. Reporting the token streams separately (43.59M student + 173.60M auxiliary response tokens for DM-OPD versus 111.07M direct student tokens for the auxiliary-free control; Tables 6, 21) confirms the objective's advantage at equal measured training compute.

Non-thinking 4B armSteps / cyclesStudent + auxiliary tokensGPU-h4K Macro8K Macro
Non-think: DM-OPD (guard off)400 steps43.59M + 173.60M67.9159.9860.22
Non-think: Extended native OPDLM480 cyc (7,680 upd)42.31M + 068.9150.7050.71
Non-think: Matched-decoder OPDLM-loss830 steps111.07M + 068.4249.0249.67
  • Two descriptive controls (Tables 6, 19, 21; Appendix H.1): Extended-compute native OPDLM runs the native block-random masking recipe in our shared infrastructure on the 61,816-prompt corpus, preserving its released hyperparameters (16 minibatches of 8 per 128-rollout cycle, LR 10−510^{-5}; 227.32M total packed sequence tokens across updates). Matched-decoder OPDLM-loss control uses our exact B4I4 decoder (guard off), all-position intermediate-token forward KL, no auxiliary model, and our matched student optimizer settings (1 batch of 128 per 128 rollouts, LR 2×10−62\times 10^{-6}). Skipping auxiliary rollouts and updates cuts step time from 76.40 s to 37.09 s, giving the matched-decoder control 830 student updates (111.07M student tokens) in 68.42 GPU-hours, yet it trails DM-OPD by 10.96 pp at 4K (Figure 10).
AuthorsOpenReview ↗

Response to Reviewer P5Qe (2/2)

2. Comparability, Evaluation Audit, and OPDLM-MATH Budget Resolution

Our revised comparisons match each model's training and evaluation modes (Table 1), include three benchmarks that are completely unmonitored during training (Appendix B.5), and resolve the specialist OPDLM-MATH-4B-Thinking gap at a 32K generation budget where OPDLM-MATH approaches its published scores (42.08% AIME24, 90.60% MATH500) while generalist DM-OPD reaches 71.25% and 96.40% (Tables 1, 18).

  • Specialist generation budget (Table 18; Appendix G.1): Under the shared 8K protocol, OPDLM-MATH scores 30.63%, 25.00%, and 85.90% on AIME24, AIME25, and MATH500 because over 80% of its AIME responses truncate at the 8K cap. At 32K it reaches 42.08%, 44.17%, and 90.60%, matching its published AIME24/MATH500 results. On the identical 32K prompt panel, our generalist 4B student reaches 71.25%, 64.58%, and 96.40%, resolving the discrepancy without inferring an unreported budget. The ±7.23 in Table 18 is a question-clustered standard error reflecting variation across problems. On the fixed 30 AIME24 problems, the estimated sampling standard error of our eight-draw reproduction mean is 1.96 percentage points (Appendix G.1).

  • Monitoring and prompt overlap (Appendices B.5, E.1): Yes—the reduced 2,977-response training monitor reuses original questions from the five mathematics/science benchmarks. The original 4B checkpoint (step 137) stopped at a wall-clock cutoff while still improving, so we continued training under the same settings to a fixed 400-update budget, set before its endpoint evaluation. Some accelerated runs stop on monitoring saturation (Table 8). Final evaluation uses the full panels without repetition cutoffs, and MMLU, IFEval, and HumanEval were never monitored (on these three unmonitored benchmarks, 4B non-thinking DM-OPD averages 75.37% at 4K vs. 59.83% and 58.93% for the controls; Table 19). The separate prompt-overlap audit finds four MATH500 matches and none in the other seven benchmarks; excluding those four shifts macro accuracy by at most 0.0524 points (Appendices B.5, E.1).

3. Quality–Speed Frontier and Deployment Positioning vs. AR

B4I3 is our demonstrated lower-latency operating point relative to AR. At 4K in matched thinking evaluation, it retains near-teacher mathematical reasoning (92.72% GSM8K and 75.15% MATH500 vs. 93.50% and 77.80% for AR; Table 2) while cutting fixed-4K BS1 latency from 15.564 s to 13.548 s (1.15x speedup; Table 29). The original latency figures (Table 13 of the submission) were inflated by performance bugs in our serving engine that are now fixed; the figures here are re-measured on the fixed engine. Faster schedules retain a trade-off across the full eight-benchmark panel, while B4I4 remains the quality-focused configuration (0.86x AR speed at 4K; 70.90% 8K macro accuracy vs. 71.62% for AR; Table 1).

At nominal TPF 2, widening the block from B4I2 to B8I4 improves all five mathematics/science benchmarks—lifting MATH500 from 69.85% to 72.80% and GSM8K from 84.14% to 87.19% (Table 2)—while retaining a 1.67x BS1 speedup at 4K (Table 30). Tables 29–30 report the full measured trade-off, supporting parallel block refinement when single-request latency is prioritized and exposing remaining full-panel gaps.

Because these accelerated schedules use a fixed number of passes per block, integrating adaptive pass allocation into our decoder-matched training is a complementary next step for concentrating refinement compute on harder blocks.

Reviewer_Ti6GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes DM-OPD, which distills an AR teacher into a block diffusion LM by minimizing the KL between the student’s completed-block text distribution and the teacher’s, on student-generated prefixes. It argues that existing objectives (masked ELBO, OPDLM’s token-level on-policy KL) can have optima that miss an attainable target distribution, and shows this with a toy two-field example. To make the objective trainable, it uses a score-function gradient, an auxiliary AR model that estimates the student’s text likelihood, a dependency-based credit split (full block feedback for intermediate steps, suffix feedback for the final step), and a stochastic keep-or-replace decoder. With Qwen3-1.7B/4B, it beats SDAR and OPDLM on 8 benchmarks at 4K/8K limits. A controlled 1.7B ablation shows +5.4 points over OPDLM’s objective.

Critical strengths
  • Clear problem and principled fix. The toy examples (Prop. 1, A.2, A.3) show that lowering token-level loss can move the completed-text distribution away from an attainable target. The completed-text KL has the teacher as its optimum, and the credit decomposition comes with unbiasedness proofs under exact density. This gives the paper a real conceptual contribution, not just a recipe.
  • Large, consistent gains with a controlled ablation. The 1.7B student beats SDAR and OPDLM on all 8 benchmarks at both limits, and the 4B student does so at 8K. The ablation holds decoder, data, and seeds fixed and gives paired CIs (+5.44 [3.84, 7.08]). It also shows that OPDLM’s loss falls while accuracy drops.
Critical weaknesses

Main comparison is confounded, and evidence for the objective is thin.

  • Table 1 differs from the baselines in decoder (keep-or-replace vs. threshold), in thinking-mode training (OPDLM was trained without thinking and only evaluated with it), and in the optional guard.
  • The only controlled test of the objective is one run per arm, one seed, 1.7B, 180 updates, on a monitoring panel.
  • Checkpoints are chosen by monitoring accuracy on a subset of the same five benchmarks they are reported on.
  • OPDLM’s objective ending below its own initialization is surprising and suggests it may be under-tuned. The authors’ own OPDLM-MATH reproduction (31% vs. 43.3% reported) adds to this concern.

Efficiency claims are overstated and costs are high.

  • The headline results are at TPF 1.0 (B4I4), which gives no parallelism.
  • At 2 iterations, quality drops sharply (MATH500 73→41), and a threshold decoder beats the student on 4 of 5 benchmarks.
  • Batch-1 latency is worse than AR (Table 13).
  • The auxiliary model needs about 3.5× the student’s tokens (170M vs. 48M), so total training cost is much higher than OPDLM’s.
  • Gaps to AR remain large on HumanEval and IFEval.

Gap between theory and practice.

  • The mismatch evidence is a contrived 2-field tabular example with a specific decoder.
  • The unbiased-gradient guarantee needs exact density and no clipping, but the real recipe uses estimated density, clipping, whitening, and dual clipping (Prop. 3(iii) concedes this).
  • Auxiliary density accuracy was never measured (“not measured directly”).
  • Variance of the estimator is not analyzed.
  • The guard is ad hoc and shows no measurable endpoint effect.
Questions for authors
  • Is the monitoring panel (used for checkpoint selection) drawn from the same questions as the reported benchmarks?
  • How close does the auxiliary get to the student’s true text distribution during training (e.g., held-out NLL gap, or aux-vs-student agreement)?
  • How were OPDLM-objective (B) hyperparameters tuned, and why does accuracy end below initialization?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Ti6G (1/2)

We thank Reviewer Ti6G for the questions on objective attribution, evaluation overlap, estimator behavior, and generation quality.

Updates during rebuttal. Compared with the original submission, the DM-OPD scores and figures reflect extended training and new 4B evaluations completed during rebuttal. The AR reference was re-evaluated on the same H200 hardware as the diffusion models to remove hardware differences, so its scores differ slightly from the original Table 1. The main tables and quality–speed charts now focus on 4B models, and the original 1.7B quality results remain in Appendix F. The original wall-clock table (1.7B, RTX 5090) is removed: those timings were inflated by several performance bugs in our serving engine, which are now fixed. The new 4B measurements use the fixed engine on an H200, with AR timed in the same engine (Appendix I). All citations below refer to the revised PDF, where key revisions are marked in blue; the original Appendices F, G, and H are now G, H, and I.

1. Extended 4B Objective Controls

At approximately 68 H200 method GPU-hours per non-thinking arm (including all auxiliary work), DM-OPD leads extended native OPDLM by +9.28/+9.51 pp and the matched-decoder OPDLM-loss control by +10.96/+10.56 pp at 4K/8K (Tables 19, 21). Separate thinking-mode controls yield +7.19–9.09 pp gains (Table 19).

Non-thinking armGPU-h4K macro8K macro
DM-OPD, guard off67.9159.9860.22
Extended native OPDLM68.9150.7050.71
Matched-decoder OPDLM-loss68.4249.0249.67

Two distinct controls (Table 6; Appendix H.1): Extended native OPDLM runs its native masking recipe and released hyperparameters on our prompt corpus, reaching 50.70% at 4K—close to the released checkpoint's 50.37% (Tables 17, 19). The matched-decoder loss control uses our B4I4 decoder and student optimizer, applying intermediate-token forward KL to all positions without an auxiliary. Neither control received an objective-specific sweep. The native control continues OPDLM's released recipe and hyperparameters; the matched-decoder control adapts the OPDLM loss to our decoder and batch configuration and shares our commonly used post-training hyperparameters (Appendix B.5).

Below-initialization accuracy: Lower intermediate-token loss need not improve the completed-text distribution—the mismatch analyzed in Section 3. The 1.7B loss control reduces token prediction loss from 7.82 to below 1 nat/token while ending at 50.08% versus 50.69% initially (Appendix H.4, Table 25). In the extended 4B loss control, the final monitoring evaluation ends above its initial MATH500 accuracy and slightly below its initial GSM8K accuracy, whereas DM-OPD improves steadily throughout training (Figure 10).

Modes and specialist baseline: Primary comparisons match training and evaluation modes (Table 1). At 32K, OPDLM-MATH reaches 42.08% AIME24 and 90.60% MATH500, close to its reported 43.3%/90.2%, while our generalist 4B student reaches 71.25%/96.40% on the same panel and cap (Table 18). The ±7.23 in Table 18 is a question-clustered standard error reflecting variation across problems. On the fixed 30 AIME24 problems, the estimated sampling standard error of our eight-draw reproduction mean is 1.96 percentage points (Appendix G.1).

2. Monitoring and Prompt Overlap

Yes—the reduced monitor shares original questions with the five mathematics/science benchmarks. The original 4B checkpoint (step 137) stopped at a wall-clock cutoff while still improving, so we continued training under the same settings to a fixed 400-update budget, set before its endpoint evaluation. Some accelerated runs stop on monitoring saturation (Table 8). Final evaluations use the full panels without repetition cutoffs, and MMLU, IFEval, and HumanEval were never monitored (Appendix B.5). A separate training-prompt audit finds four MATH500 matches and none in the other seven benchmarks; removing them shifts macro accuracy by at most 0.0524 points (Appendix E.1).

AuthorsOpenReview ↗

Response to Reviewer Ti6G (2/2)

3. Practical Estimator and Guard

Proposition 1's analytical construction isolates how step-wise token losses diverge from completed-text matching, while the 4B controls test our objective in language models. The exact gradient identity uses true text density and unclipped on-policy sampling; in practice, we fit p^ψ\hat p_\psi online and center assigned credit to remove common offsets, while PPO/dual clipping is inactive and TIS clipping is rare (Appendices B.2–B.3). Direct diagnostics confirm each component:

  • Auxiliary tracking: The current auxiliary fits identical current-student samples better than the preceding auxiliary at all seven post-initial checkpoints; every paired 95% CI excludes zero (Table 23).
  • Teacher likelihood: DM-OPD obtains 0.433/0.436 nats/token teacher NLL at 4K/8K, versus native OPDLM's 1.567/1.552 with our B4I4 decoder and 0.576/0.539 with its threshold decoder (Table 22).
  • Gradient variability: All 16 measured minibatches align positively with a disjoint reference (mean cosine 0.0747; averaged-batch cosine 0.2621; Table 24). TIS clips 0/3,781,803 events; feedback clips 0.01035% of scored tokens (Appendix H.3).
  • Guard: Used for empirical early-stage stabilization, the guard removes the initial plateau in 1.7B B4I2 while shifting paired 4B B4I4 endpoints by at most 1.20 points across 4K/8K settings (Figure 6; Table 20; Appendix H.4).

4. Quality and Parallel Generation

The completed thinking B4I4 run reaches 82.47%/89.94% HumanEval and 77.87%/78.65% IFEval at 4K/8K (Tables 1, 15). Separately, B4I3, B4I2, and B8I4 deliver 1.15×, 1.70×, and 1.67× single-request H200 speedups at 4K (Tables 29–30). The original latency figures (Table 13 of the submission) were inflated by performance bugs in our serving engine that are now fixed; the figures here are re-measured on the fixed engine.

Our students use a fixed number of passes per block; adaptive allocation is future work. Even under fixed schedules, they are ahead in 29 of 30 mathematics/science comparisons at matched fixed budgets (Table 2), with higher eight-benchmark macro at each budget: thinking 60.21/53.10/44.40 versus SDAR 51.22/47.87/42.91, and non-thinking 59.98/55.97/48.10 versus OPDLM 50.37/47.38/43.33, in B4I4/B4I3/B4I2 order (Table 17).

We clarified auxiliary scoring versus parallel generation in Section 4.2 and moved the guard formulation and curves into Section 4.6.

I thank the authors for their detailed explanations and clarifications. However, I have decided to maintain my original scores.

Reviewer_LhZXOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper introduces Distribution Matching On-Policy Distillation (DM-OPD) to convert pretrained autoregressive (AR) language models (Qwen3-1.7B/4B) into block-diffusion language models. The authors argue via a two-variable analytical toy example that matching intermediate token predictions on student trajectories (as in OPDLM) can cause mode collapse in the final completed-text distribution. Instead, DM-OPD minimizes the Reverse KL divergence between the student's completed-block text distribution and the AR teacher's distribution. Because the marginal probability of a completed text under a multi-step refinement decoder is intractable, DM-OPD trains an auxiliary AR model (p^ψ\hat{p}_\psi) on student outputs to estimate student text likelihoods, assigns dependency-based action credit across refinement steps, and uses a stochastic keep-or-replace decoding rule over 4-token blocks.

Critical strengths
  • Identifies Exposure & Objective Mismatch in Multi-Step Decoding: The paper highlights an interesting theoretical tension between minimizing step-wise conditional token losses and matching the marginal distribution induced by a multi-step commitment decoder.
Critical weaknesses
  1. The Student Operates Nearly as an L2R Autoregressive Model Rather Than a True Parallel Diffusion Model
  • Tiny Block Size (B=4B=4) and Lack of Masked Diffusion Training: The canvas size for parallel refinement is restricted to just B=4B=4 tokens (B4I4), while generation across blocks remains strictly causal and left-to-right.

  • No Parallel Speedup in Main Results: In Table 1, every reported DM-OPD result uses 4 sequential iterations to generate a 4-token block, resulting in 1.00 Tokens Per Forward and 2.0x worse single-stream latency than the base Qwen3 AR model. Thus, the headline accuracy gains come from a regime with zero parallel decoding speedup over standard AR generation.

  • Severe Degradation Under Parallel Generation: When the model is actually tasked with generating tokens in parallel, accuracy drops precipitously (e.g., AIME24 falls from 10.83% to 3.33%, and MATH500 drops from 73.35% to 41.05%). For typical diffusion models like LLaDA, Dream, or Nemotron-Labs-Diffusion, they are generally able to generate at least 22 tokens per forward while barely suffering any accuracy degradation. At TPF≈2.0\text{TPF} \approx 2.0, the existing SDAR-Chat baseline with confidence thresholding (τ=0.90\tau = 0.90, TPF=2.08\text{TPF} = 2.08) outperforms DM-OPD B4I2 on GSM8K, MATH500, and GPQA.

  1. Questionable Auxiliary AR Surrogate (p^ψ\hat{p}_\psi) and L2R Credit Assignment:
  • Using an L2R AR Model to Estimate Bidirectional Block Probabilities: To estimate the student's completed-text density, the authors train a separate full-sized AR model on student outputs and compute token-wise L2R log-ratios. At the final refinement step, action credit is assigned via a strictly left-to-right suffix sum. Imposing an L2R conditional factorization and causal suffix credit onto the diffusion student's updates risks pressuring the student to behave like a pseudo-AR model rather than leveraging true bidirectional context.

  • Massive Computational Overhead of the Auxiliary AR Model: The auxiliary AR model is updated 4 times per student update and consumes 3.5x to 4x more generated tokens than the student itself. Furthermore, there is no analytics to check how well the student AR model actually tracks the diffusion model student.

Questions for authors

Please see outlined weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer LhZX (1/2)

We thank Reviewer LhZX for the questions on accelerated (TPF > 1) quality, how block diffusion differs from left-to-right decoding, and how our auxiliary model and credit assignment work.

Updates during rebuttal. Compared with the original submission, the DM-OPD scores and figures reflect extended training and new 4B evaluations completed during rebuttal. The AR reference was re-evaluated on the same H200 hardware as the diffusion models to remove hardware differences, so its scores differ slightly from the original Table 1. The main tables and quality–speed charts now focus on 4B models, and the original 1.7B quality results remain in Appendix F. The original wall-clock table (1.7B, RTX 5090) is removed: those timings were inflated by several performance bugs in our serving engine, which are now fixed. The new 4B measurements use the fixed engine on an H200, with AR timed in the same engine (Appendix I). All citations below refer to the revised PDF, where key revisions are marked in blue; the original Appendices F, G, and H are now G, H, and I.

1. Quality at Accelerated Schedules and Larger-Block Gains

Yes, quality drops as iterations are reduced, and we state this as a limitation. Our students apply a fixed number of passes to every block, whereas confidence-threshold decoders can allocate more passes to difficult blocks at the same average TPF; integrating adaptive pass allocation is future work. Even under a fixed schedule, our 4B students lead the baselines at every matched fixed budget: across B4I4, B4I3, and B4I2 in both thinking and non-thinking modes at 4K, they are ahead in 29 of 30 mathematics/science comparisons at matched fixed budgets (Table 2). Our thinking B4I2 student (TPF 2.00) also exceeds SDAR on all five of these benchmarks under both SDAR's threshold decoder (TPF 1.87) and its full four-iteration schedule (Table 2). Threshold decoding keeps the baselines stable, but well below the AR reference: SDAR reaches 51.22% eight-benchmark macro accuracy at four iterations and 51.18% with threshold decoding, compared with 62.62% for AR and 60.21% for our B4I4 student (Table 17).

The original latency figures (Table 13 of the submission) were inflated by performance bugs in our serving engine that are now fixed; the figures here are re-measured on the fixed engine.

The table below summarizes 4B reasoning accuracy at 4K under matched prompt modes (Table 2; complete eight-benchmark scores in Appendix G, Table 17) alongside batch-size-1 H200 speedups (Appendix I, Tables 29–30).

Model and modeTPF / BS1 speedupAIME24GSM8KMATH500GPQA
Thinking: Qwen3-4B AR1.00 / 1.00x22.9293.5077.8045.08
Thinking: SDAR threshold 0.91.87 / —7.9289.6769.5034.22
Thinking: DM-OPD B4I41.00 / 0.86x22.4093.2778.4041.29
Thinking: DM-OPD B4I31.33 / 1.15x20.7392.7275.1540.78
Thinking: DM-OPD B4I22.00 / 1.70x17.9292.4072.9036.36
Non-thinking: Qwen3-4B AR1.00 / 1.00x23.9690.1183.7541.16
Non-thinking: OPDLM threshold 0.91.88 / —12.2986.1171.9030.05
Non-thinking: DM-OPD B4I31.32 / 1.15x20.3188.3179.1040.09
Non-thinking: DM-OPD B4I21.99 / 1.70x11.2584.1469.8535.61
Non-thinking: DM-OPD B8I41.93 / 1.67x11.3587.1972.8036.24

At nominal TPF = 2.0, widening the block from B4I2 to B8I4 lifts 4K non-thinking macro accuracy from 48.10% to 49.91% (+2.95 pp MATH500, +3.05 pp GSM8K, +7.32 pp HumanEval; Table 17).

At 1.7B, extending B4I2 training from step 503 to step 991 raises GSM8K accuracy from 74.41% to 81.94% and MATH500 from 41.05% to 51.80%, surpassing the SDAR threshold baseline on both benchmarks (76.35% and 51.15%), while SDAR remains higher on GPQA (26.01% vs. 22.10%; Appendix F, Table 12). At 4B, our B4I3 students outperform the baselines' full four-iteration schedules on all five mathematics/science benchmarks (Table 2), with a 1.15x single-request speedup over AR (Table 29).

AuthorsOpenReview ↗

Response to Reviewer LhZX (2/2)

2. Block Diffusion and Non-Left-to-Right Generation

DM-OPD uses block diffusion, the same model class used by LLaDA2.0, SDAR, and OPDLM. Blocks advance sequentially, while the tokens within each block are generated and revised jointly through multiple parallel refinement steps. This is different from token-by-token AR decoding, which commits each token before generating the next.

Our saved traces directly show non-left-to-right decoding. In the AIME24 trace (step 2440, block 6; Table 3), Iteration 1 proposes [" a", " than", " of", " candy"]. Iteration 2 changes the earlier second token to " bag" while keeping the later " of candy" tokens fixed. All four positions are committed together. Table 3(b) also shows simultaneous two-token revisions in blocks 6 and 7. Tables 4–5 show complete 256-token traces: the second parallel pass revises 62/256 tokens in the thinking-trained model and 61/256 in the non-thinking-trained model.

These traces distinguish the actual generation process from strict left-to-right token commitment. B4I4 having one token per forward describes its compute budget, not an AR generation order; the accelerated B4I3/B4I2 results and wider-block B8I4 results are reported in §1 above.

Converting a pretrained AR checkpoint directly into a block-diffusion generator through OPD, without masked-diffusion pretraining, is an intentional part of our contribution.

3. Why Auxiliary AR Scoring and Eq. (8) Preserve Bidirectional Generation

The auxiliary's chain-rule factorization scores a block after all II bidirectional refinement steps complete; intermediate refinement actions receive full-block feedback across all BB tokens, and only the terminal step uses causal suffix credit along the dependency graph.

The student's completed-text probability sums over compatible refinement paths, pθtext(xb∣hb)=∑τb:F(τb)=xbPθ(τb∣hb)p_\theta^{\mathrm{text}}(x_b\mid h_b)=\sum_{\tau_b:F(\tau_b)=x_b}P_\theta(\tau_b\mid h_b), so its intermediate logits do not directly provide that marginal. By the probability chain rule, any completed-text distribution admits an AR factorization. We fit p^ψ\hat p_\psi to student completions and use di=log⁡qϕ(xi∣c,x<i)−log⁡p^ψ(xi∣c,x<i)d_i=\log q_\phi(x_i\mid c,x_{<i})-\log\hat p_\psi(x_i\mid c,x_{<i}) as feedback (Section 4.2). This scores completed text while preserving the student's parallel, bidirectional decoder.

Intermediate actions can affect all final block positions and receive full-block feedback. At the final parallel draw, earlier-prefix likelihood terms have zero expected score product with a later action; retaining the suffix preserves the expected gradient (Eq. 8, Appendix A.6). This is dependency-based credit, not an imposed left-to-right generation order.

  • Auxiliary fit and cost (Tables 19, 21, 23): The current auxiliary fits identical current-student samples better than the preceding auxiliary at all seven post-initial checkpoints (paired 95% intervals exclude zero). At approximately 68 H200 GPU-hours including auxiliary work, DM-OPD achieves 59.98% 4K macro accuracy versus 50.70% for extended-compute native OPDLM and 49.02% for the auxiliary-free matched-decoder loss control.

I would like to thank the authors for their response, which has cleared up a lot of things for me. Given the late stage in the review process, I do not expect the authors to carry out additional evaluations.

The authors have provided parallel decoding runs confirming that the trained DLM achieves generation speedups relative to prominent AR adaptation baselines. They have also clarified that the model does not generate strictly left-to-right. This have substantially improved the paper's soundness in my opinion.

However, one major limitation remains unclear to me:

It is still not entirely obvious how using on-policy gradients driven by an auxiliary AR surrogate model provides a sound, stable method for training a multi-step diffusion model. The potential distribution mismatch between the auxiliary approximation and the true student distribution is not fully addressed (I expect that the reverse KL does a lot of heavy lifting there). I would like the revision to include a more complete discussion of this particular aspect.

That said, I respect the methodological boldness and novelty of the approach and will update my score accordingly.

Thank you for the constructive follow-up. We agree that the reverse-KL structure does much of the work here: the student gradient is an expectation over its own multi-step sampling trajectories, so the auxiliary is queried on the same student text distribution it is continually fitted to. At a fixed student, auxiliary maximum-likelihood training minimizes KL(pθtext∥p^ψ)\mathrm{KL}(p_\theta^{\mathrm{text}}\|\hat p_\psi). The differentiated action probabilities still come from the student's actual diffusion trajectory.

We initialize the auxiliary from the teacher, so teacher–auxiliary feedback starts near zero; subsequent fitting adapts it to student outputs (Appendix B.2). It receives four optimizer updates per student update using separate auxiliary samples, before it scores the policy batch. A related principle appears in image-generation distillation: DMD and DMD2 train an auxiliary score model on current generator outputs, and DMD2 improves stability by updating it more frequently (Yin et al., NeurIPS 2024).

The practical update is not exactly unbiased (Proposition 3(iii)). Appendix A.7, Eq. 41 bounds the auxiliary-induced gradient error using log-density errors and trajectory-score magnitudes averaged over student rollouts. Thus, auxiliary fitting targets distributional mismatch on the same distribution where estimation error enters the policy update. Exact density recovery is stronger than necessary: positive alignment ⟨g,g^⟩>0\langle g,\hat g\rangle>0 between the ideal and approximate update directions suffices for improvement at a sufficiently small step (Eq. 42).

Empirically, Appendix H.2 (Table 23) tracks the 4B non-thinking, guard-on B4I4 run using fresh student samples from 128 held-out prompts. At all seven post-initial checkpoints, the current auxiliary assigns lower NLL to the same current-student samples than the preceding checkpoint's auxiliary, with all paired 95% bootstrap intervals excluding zero. From steps 40 to 400, teacher NLL on student outputs also falls monotonically from 0.6466 to 0.3256 nats/token, indicating steady progress under the coupled optimization.

Together, these observations indicate that residual approximation error does not prevent effective distillation. Across our final B4I4 evaluations, 8K macro accuracy reaches 70.90% versus 71.62% for AR in thinking mode, and 60.22% versus 61.82% for the guard-off non-thinking model (Table 1).

Section 4.2 presents the completed-text feedback construction. Appendix A.7 analyzes auxiliary-density error, and Appendices H.2–H.3 report the tracking and update-consistency diagnostics discussed above.

Reviewer_eogJOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DM-OPD, an approach for distilling autoregressive language models into diffusion language models by matching the distributions of completed text rather than intermediate token predictions.

The authors demonstrate that minimizing standard masked-diffusion or on-policy token-level distillation objectives can produce a distribution of completed text that differs substantially from the teacher's, even when the decoder can represent the target distribution exactly. This mismatch arises because optimizing intermediate token predictions does not necessarily optimize the final text distribution induced by the model's iterative decoding process. They propose minimizing the reverse KL divergence between the student decoder's completed-text distribution and an autoregressive teacher. Their practical method combines an auxiliary autoregressive model for estimating student likelihoods, dependency-based action credit assignment, and a stochastic keep-or-replace decoder.

Experiments using Qwen3-1.7B and Qwen3-4B initialization show substantial improvements over SDAR and OPDLM across eight benchmarks. The authors also provide controlled objective ablations and investigate generation quality at different refinement budgets.

Critical strengths
  1. A well-motivated distribution-level distillation objective. I found the analysis demonstrating that intermediate-prediction matching can move the student's completed-text distribution away from an attainable teacher distribution particularly interesting and well done. The authors provide a construction and gradient derivation, a principled motivation for optimizing the decoder-induced output distribution directly. The dependency-based action-credit formulation is also a useful contribution.

  2. Strong empirical improvements with meaningful objective ablations. The paper reports substantial improvements over SDAR and OPDLM across eight benchmarks at both 1.7B and 4B scales. The authors compare DM-OPD against OPDLM's objective while holding the decoder and training setup fixed. The reported 5.44-point improvement on the monitoring panel supports the claim that the completed text objective contributes to the gains beyond differences in the decoder or training recipe.

Critical weaknesses
  1. The practical distribution-matching mechanism requires more validation. The theoretical gradient estimator they proposed assumes access to the student's exact completed text distribution, but the practical method substitutes an auxiliary autoregressive density model and applies clipping and advantage normalization. The authors appropriately acknowledge that the implemented estimator is therefore not generally unbiased. However, I found no direct evaluation of how accurately the auxiliary approximates the student distribution or whether training actually reduces the student–teacher output-distribution discrepancy. The experiments primarily demonstrate improvements in benchmark accuracy, which does not necessarily establish distribution matching. Also in Appendix G the main DM-OPD run uses a different replay embedding path from the other ablation runs, potentially complicating the interpretation of the auxiliary-density ablation.

  2. Evaluation independence and compute-efficiency claims require further evidence. The authors monitor training using 2,977 responses drawn from the same five benchmark suites used in the final evaluation, and at least one checkpoint is selected based on a plateau in monitoring accuracy. Although the final evaluation uses a larger response panel, it is unclear whether checkpoint selection is independent of the final benchmark questions.

I also feel that the compute efficiency comparison could be more complete. Appendix B.4 reports student and auxiliary training-token counts, but not matched total training FLOPs or GPU-hours. This is important because DM-OPD requires four additional auxiliary AR-model updates per student update, whereas the controlled OPDLM-objective baseline does not train an auxiliary model. Thus, the controlled ablation establishes an improvement at matched student-update counts but not matched distillation compute. Appendix H shows that the highest-quality 1.7B configuration has approximately twice the single-request latency of autoregressive Qwen3 and only 1.03× its batched throughput. Corresponding wall clock measurements for the stronger 4B results are important to include.

Questions for authors
  1. Can the authors provide evidence that the auxiliary autoregressive model accurately estimates the student's completed-text distribution, for example through held-out likelihood or sample-distribution diagnostics? I would also be interested in whether DM-OPD measurably improves student–teacher distributional alignment or diversity beyond benchmark accuracy. Additionally, could the authors clarify whether the different replay embedding path used in run C affects the controlled auxiliary density ablation?

  2. The paper uses a 2,977-response monitoring panel drawn from the same five benchmark suites used for final evaluation. Do the monitoring and final evaluation sets share original questions? Results using a disjoint validation set would strengthen the evaluation. Please also clarify whether the training prompt corpus was checked for overlap with the evaluation benchmarks.

  3. Appendix B.4 reports student and auxiliary training-token counts, but I found no matched-total-FLOPs or GPU-hour comparison with OPDLM. This is important because DM-OPD performs four additional auxiliary AR-model updates per student update, whereas the controlled OPDLM-objective baseline does not train an auxiliary model. Can the authors report total distillation FLOPs or GPU-hours, including student rollouts, student updates, teacher scoring, and auxiliary training? Ideally, I would like to see benchmark performance as a function of total distillation compute for DM-OPD and the matched-decoder OPDLM-objective baseline.

    Additionally, Appendix H shows that the 1.7B B4I4 configuration has approximately twice the single-request latency of autoregressive Qwen3, with nearly identical batched throughput. Could the authors provide corresponding end-to-end wall-clock measurements for the 4B models, particularly at the 4K and 8K response limits, to establish the practical quality–efficiency tradeoff? Peak HBM/memory usage of reported methods under a fixed input/output sequence length protocol would also helpful from a reproducibility standpoint.

2 comments
AuthorsOpenReview ↗

Response to Reviewer eogJ (1/2)

We thank Reviewer eogJ for the questions on auxiliary tracking, distributional alignment, evaluation overlap, matched compute, and 4B serving performance.

Updates during rebuttal. Compared with the original submission, the DM-OPD scores and figures reflect extended training and new 4B evaluations completed during rebuttal. The AR reference was re-evaluated on the same H200 hardware as the diffusion models to remove hardware differences, so its scores differ slightly from the original Table 1. The main tables and quality–speed charts now focus on 4B models, and the original 1.7B quality results remain in Appendix F. The original wall-clock table (1.7B, RTX 5090) is removed: those timings were inflated by several performance bugs in our serving engine, which are now fixed. The new 4B measurements use the fixed engine on an H200, with AR timed in the same engine (Appendix I). All citations below refer to the revised PDF, where key revisions are marked in blue; the original Appendices F, G, and H are now G, H, and I.

1. Auxiliary Tracking, Teacher Likelihood, and Replay Embedding

  • Online auxiliary tracking (Table 23; Appendix H.2): On fresh samples from the current 4B non-thinking, guard-on B4I4 student, we compare the current auxiliary p^ψt\hat p_{\psi_t} with the preceding checkpoint's auxiliary p^ψt−Δ\hat p_{\psi_{t-\Delta}}. The current auxiliary has lower held-out NLL at all seven post-initial checkpoints, with every paired 95% CI excluding zero, directly verifying adaptation to the moving target.

  • Token-level alignment diagnostic (Table 23; Appendix H.2): On those same student samples, the teacher-minus-auxiliary NLL gap decreases from 0.265 to 0.035 nats/token between steps 40 and 400. Because expected log⁡(p^ψt/qϕ)\log(\hat p_{\psi_t}/q_\phi) over scored token–prefix pairs equals mean conditional KL(pt∥qϕ)−KL(pt∥p^ψt)\mathrm{KL}(p_t\|q_\phi)-\mathrm{KL}(p_t\|\hat p_{\psi_t}), this gap estimates a lower bound on the mean conditional student–teacher KL of the scored tokens.

  • Held-out teacher NLL (thinking-trained checkpoints; Table 22): On 128 held-out prompts (512 responses/cell), DM-OPD (step 401, Jacobi B4I4) achieves 0.433 / 0.436 nats/token teacher NLL at 4K/8K, versus 1.567 / 1.552 (Jacobi B4I4) and 0.576 / 0.539 (threshold 0.9) for extended-compute native OPDLM (step 670), showing higher teacher likelihood under both decoders.

  • Replay embedding path (Appendices C.4, H.4): The remark in the submitted paper was an outdated implementation note left in place after we verified equivalence. The two paths batch the same token-embedding and timestep-conditioning lookups per sequence or per concatenated pack. Across 8 FSDP ranks, they produce identical embedding outputs, relative L1L_1 gradient difference 4.118e-8, and gradient cosine similarity 1.00000000. This implementation detail therefore does not change the interpretation of C$-$F as the student-density ablation, and the revised manuscript replaces the obsolete caveat with this verification.

2. Monitoring Overlap and Training-Prompt Audit

Yes—the reduced training monitor shares original questions with the five mathematics/science benchmarks. The original 4B checkpoint (step 137) stopped at a wall-clock cutoff while still improving, so we continued training under the same settings to a fixed 400-update budget, set before its endpoint evaluation. Some accelerated runs stop on monitoring saturation (Table 8). Final evaluation uses the full panels without repetition cutoffs, and MMLU, IFEval, and HumanEval were never monitored (Appendix B.5). A separate direct-prompt-overlap audit finds four MATH500 matches and none in the other seven benchmarks; excluding those four shifts macro accuracy by at most 0.0524 points (Appendix E.1).

Disjoint-set result (benchmarks excluded from monitoring; Table 19): Across the three unmonitored benchmarks (MMLU, IFEval, and HumanEval), mean 4K non-thinking accuracy is 75.37% for DM-OPD, 59.83% for extended native OPDLM, and 58.93% for the matched-decoder loss control.

AuthorsOpenReview ↗

Response to Reviewer eogJ (2/2)

3. Complete GPU-Hour and Student/Auxiliary Token Accounting

At approximately 68 H200 GPU-hours per 4B non-thinking arm, guard-off DM-OPD achieves 59.98% / 60.22% macro accuracy at 4K/8K, versus 50.70% / 50.71% for extended-compute native OPDLM and 49.02% / 49.67% for the matched-decoder loss control (Tables 19, 21). This includes student and auxiliary rollouts, teacher/auxiliary scoring, and optimizer updates; evaluation and reward-wait overhead are excluded consistently. DM-OPD uses 43.59M student and 173.60M auxiliary response tokens (400 updates), whereas the auxiliary-free matched-decoder control uses 111.07M student tokens across 830 updates yet trails by 10.96 points at 4K (Table 21; Figure 10).

4. Same-Engine 4B Wall-Clock Latency, Throughput, and Memory

The original latency figures (Table 13 of the submission) were inflated by performance bugs in our serving engine that are now fixed; the figures here are re-measured on the fixed engine. On one H200 using 512-token prompts and fixed 4K/8K outputs with stopping disabled, we measure the following BS1 latencies (seconds), with speedups relative to the AR reference from the same configuration:

Decoder4K latency4K speedup8K latency8K speedup
AR, B4I4/B4I3 matrix15.5641.00x31.8421.00x
DM-OPD B4I418.0010.86x37.0260.86x
DM-OPD B4I313.5481.15x27.8921.14x
AR, B4I2/B8I4 matrix15.8511.00x32.3761.00x
DM-OPD B4I29.3131.70x19.1931.69x
DM-OPD B8I49.4741.67x19.7481.64x

Appendix I (Tables 29–30) gives BS1/8/32 throughput and maximum allocated CUDA memory. The 71.396–71.944 GiB figures include the preallocated KV pool, model, and graph allocations under mem_fraction_static=0.5, describing a common configured memory budget rather than each model's minimum memory requirement. We use a BF16 body, FP32 LM head, two warmups, and the median of three timed repetitions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary for the Area Chair (1/2)

For the Area Chair's convenience, we summarize the primary review concerns, the evidence provided in our individual responses, and the corresponding manuscript updates.

Updates during rebuttal. Compared with the original submission, the DM-OPD scores and figures reflect extended training and new 4B evaluations completed during rebuttal. The AR reference was re-evaluated on the same H200 hardware as the diffusion models to remove hardware differences, so its scores differ slightly from the original Table 1. The main tables and quality–speed charts now focus on 4B models, and the original 1.7B quality results remain in Appendix F. The original wall-clock table (1.7B, RTX 5090) is removed: those timings were inflated by several performance bugs in our serving engine, which are now fixed. The new 4B measurements use the fixed engine on an H200, with AR timed in the same engine (Appendix I). All citations below refer to the revised PDF, where key revisions are marked in blue; the original Appendices F, G, and H are now G, H, and I.

  1. Quality under parallel decoding and practical efficiency (LhZX, Ti6G, eogJ, P5Qe). Reviewers questioned the B4I4 headline at one token per forward, accuracy drops at faster schedules, and gaps to AR. We explicitly state that accuracy drops at fewer iterations because our current students use a fixed number of passes per block, leaving adaptive pass allocation to future work. Within this fixed-pass regime, our 4B students at 4K are ahead in 29 of 30 mathematics/science comparisons at matched fixed budgets across B4I4, B4I3, and B4I2 in both modes (Table 2) and achieve higher eight-benchmark macro accuracy at every budget (Table 17). Completing the 4B thinking B4I4 run raises 8K macro accuracy to 70.90% versus 71.62% for AR, including 89.94% on HumanEval and 78.65% on IFEval (Tables 1, 15). At 4K on a single H200, B4I3 outperforms full-iteration diffusion baselines on all five mathematics/science benchmarks while achieving a 1.15× single-request speedup over AR, and B4I2 achieves 1.70× (Tables 2, 29–30). Longer-trained 1.7B B4I2 also lifts MATH500 from 41.05% to 51.80% and GSM8K from 74.41% to 81.94% (Table 12). The replies provide matched quality comparisons and measured latency, throughput, and memory (Tables 1–2; Appendices F, G, I).

  2. Attribution, auxiliary compute, and baseline comparability (all four reviewers). We added extended-compute native OPDLM and an auxiliary-free matched-decoder OPDLM-loss control under matched training and evaluation modes (Tables 6, 19, 21; Appendix H.1). At approximately 68 H200 GPU-hours per 4B non-thinking arm (including all auxiliary rollouts, updates, and scoring), guard-off DM-OPD reaches 59.98% 4K macro accuracy versus 50.70% and 49.02% for the controls, even though the auxiliary-free matched-decoder control receives 2.55×2.55\times more student response tokens (Table 21; Figure 10). For the OPDLM-MATH reproduction concern raised by Ti6G and P5Qe, extending the generation budget to 32K yields AIME24/MATH500 scores (42.08% / 90.60%) matching its published results, while generalist DM-OPD reaches 71.25% / 96.40% on the identical 32K panel (Table 18; Appendix G.1).

AuthorsOpenReview ↗

Summary for the Area Chair (2/2)

  1. Auxiliary fidelity and practical estimator behavior (eogJ, Ti6G, LhZX). We report held-out auxiliary tracking, frozen-teacher likelihoods, frozen-checkpoint gradient/clipping diagnostics, and replay-embedding equivalence checks (Tables 22–24; Appendices C.4, H.2–H.4). Across all seven post-initial checkpoints of the 4B non-thinking guard-on B4I4 run, the current auxiliary fits fresh current-student samples strictly better than the preceding checkpoint's auxiliary (all paired 95% CIs exclude zero), and the teacher-minus-auxiliary NLL gap—an estimated token-level KL lower-bound diagnostic—decreases from 0.265 to 0.035 nats/token (Table 23; Appendix H.2). The self-consistency guard removes the initial plateau in 1.7B B4I2 and shifts paired 4B B4I4 endpoints by at most 1.20 macro points across 4K/8K settings; we present these as empirical findings without claiming exact optimum preservation (Figure 6; Table 20; Appendices B.2, H.4).

  2. Whether the method remains block diffusion (LhZX). We clarified that DM-OPD targets block diffusion, as used by LLaDA2.0, SDAR, and OPDLM: blocks advance sequentially, while tokens within a block are jointly generated and revised rather than committed one at a time. The auxiliary scores completed text only after refinement concludes. Intermediate actions receive full-block feedback, and suffix credit at the terminal step follows the causal dependency graph (Eq. 8; Appendix A.6). Saved B4I2 traces directly demonstrate parallel revisions, including earlier positions changing after later positions have settled (Tables 3–5; Appendix B.1).

  3. Evaluation independence and overlap (Ti6G, eogJ, P5Qe). We clarified that the training monitor reuses original questions from the five mathematics/science benchmarks, whereas final evaluations use the full panels without repetition cutoffs and MMLU, IFEval, and HumanEval are unmonitored (Table 8; Appendix B.5; on this unmonitored subset, DM-OPD reaches 75.37% mean 4K non-thinking accuracy vs. 59.83% and 58.93% for the 4B controls, Table 19). The training-prompt audit finds four MATH500 matches and none in the other seven benchmarks; removing those four changes macro accuracy by at most 0.0524 points (Appendix E.1).

Other additions show that widening the block from B4I2 to B8I4 improves quality at equal nominal parallelism (1.67× speedup at 4K; Tables 2, 17, 30), while identifying fixed passes per block as a current limitation and adaptive pass allocation as future work. Main-text revisions remain targeted, with expanded ablations and supporting records documented in the appendices.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.