acceptodds
Under review as a conference paper at ICLR 2027

Distilling Language Knowledge Across Generative Processes

Abstract

Diffusion language models enable parallel text generation. Prior work has shown that distillation from pretrained autoregressive (AR) models offers a promising route to acquiring strong language capabilities. However, the causal-prefix supervision used in prior work does not fully account for the mismatch between the two generation processes: an AR teacher predicts from a left prefix, while a diffusion student reconstructs missing text using observed context on both sides. To address this mismatch, we introduce CrossDLM, a framework for distilling language knowledge across these generative processes. CrossDLM uses the teacher's sequence-level scores to evaluate candidate tokens in the context of subsequent text, providing supervision for bidirectional reconstruction. This supervision incorporates visible evidence while accounting for uncertainty in unresolved content. Shared teacher computation across token alternatives and aggregation over a small number of student-generated completions make this supervision practical. With on-policy self-distillation, CrossDLM converts pretrained AR checkpoints into DLMs using approximately 0.10B training tokens per model. It outperforms causal-prefix distillation across a range of benchmarks and model families.

Primary area:
generative models
Keywords:
Diffusion Language Models, Knowledge Distillation, On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iWyGOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes CrossDLM, a framework for distilling autoregressive language models into diffusion language models using supervision aligned with partially observed text. It combines approximate suffix scoring with importance-weighted aggregation over student-generated completions and improves performance over causal-prefix distillation across multiple benchmarks and model families.

Critical strengths
  1. Clear motivation. The paper identifies a concrete supervision mismatch: the teacher predicts from a left prefix, whereas the student reconstructs missing tokens using context on both sides. It also distinguishes observed evidence from unresolved content that has only been provisionally completed, making the intended improvement precise.

  2. A clear connection between theory and implementation. The formulation establishes conditional teacher marginals as the desired supervision targets, and the practical algorithm provides a principled approximation through local scoring and completion aggregation.

  3. Meaningful empirical improvements. The method achieves substantial gains on several tasks, particularly mathematical reasoning, with benefits observed across multiple model families. The matched backbones and training configurations used for OPDLM and CrossDLM reduce confounding from differences in model size, training data, and training budget.

Critical weaknesses
  1. Limited validation at larger block sizes. The main experiments use four-token blocks, leaving it unclear whether the method remains effective when more tokens are generated in parallel. Larger blocks introduce stronger dependencies among unresolved positions and may make the proposed marginal estimation less reliable.

  2. Missing comparisons with the original autoregressive models. The paper does not provide a comprehensive comparison against the original AR models in terms of both task performance and generation speed. Consequently, it remains unclear how much capability is retained during conversion and whether the resulting models offer a favorable quality–speed trade-off.

Questions for authors

Evaluate several larger block sizes and report task accuracy, generation latency, and throughput alongside the original AR models and OPDLM to establish the practical quality–speed trade-off.

3 comments

We thank the reviewer for highlighting the practical quality–speed trade-off at larger block sizes. Below, we report the corresponding scaling and source-model comparisons.

W1: larger blocks and the quality–speed trade-off

To test scaling directly, we trained two matched B16 OPDLM/CrossDLM pairs rather than decoding B4 checkpoints with larger blocks:

SeedDatasetOPDLMCrossDLMGain
42GSM8K71.2779.45+8.19
42MATH-50050.8062.60+11.80
10086GSM8K72.1083.17+11.07
10086MATH-50048.6063.40+14.80

Both seeds show consistent gains on both reasoning tasks. To isolate the quality–parallelism trade-off within diffusion decoding, we then changed only the denoising schedule: d=16 uses 16 forwards per 16-token block (about one token per forward), whereas d=8 uses eight (about two tokens per forward). Across 3,638 responses, wall time falls from 984 to 603 s (1.63x aggregate throughput), while CrossDLM remains +5.76/+7.40 points above OPDLM on GSM8K/MATH-500. This is a schedule-level speedup, not a CrossDLM-over-OPDLM or CrossDLM-over-AR latency claim.

W2: comparison with the source AR model

We appreciate the request for a direct source-AR reference. On terminal Qwen3.5-4B, cap-aware Source AR / CrossDLM is 92.42 / 91.05 on GSM8K, 79.40 / 74.20 on MATH-500, and 74.11 / 76.31 on MMLU. Thus conversion is not universally lossless, but CrossDLM preserves most source quality while enabling blockwise decoding. In the Qwen3-4B B16 control, it also recovers approximately 43% of OPDLM's conversion loss on both GSM8K and MATH-500.

For speed, an implementation-normalized Qwen3-4B H100 benchmark fixes both prompt and output to 128 tokens on the same PyTorch/SDPA/BF16 stack. B16/d8 CrossDLM needs 64 forwards and 2.991 s, whereas raw AR needs 128 decode steps and 4.878 s with the standard Hugging Face KV cache (5.571 s without it), giving 1.63x/1.86x speedups. This measures the benefit of lower sequential depth. On the separately optimized Qwen3.5 serving stack, raw AR is currently faster (4.240 versus 5.863 s), consistent with the greater maturity of AR infrastructure such as vLLM/SGLang. We therefore frame the contribution as improved adaptation with algorithmic parallelism, while optimized DLM serving remains a future direction.

Thank you to the authors for the detailed response and clarifications. In light of the rebuttal, I have raised my Soundness score, while keeping my Rating unchanged.

Thank you for your thoughtful consideration and for updating your assessment! We appreciate your feedback.

Reviewer_fsCNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CrossDLM, a framework for distilling autoregressive language models into diffusion language models. It addresses the mismatch between causal-prefix teacher supervision and the partially observed, bidirectional context available to the diffusion student. CrossDLM targets the teacher’s conditional token marginals by combining exact current-token probabilities with a gradient-based approximation of suffix likelihood changes, then aggregating student-generated completions through importance weighting. Experiments report improvements over matched OPDLM baselines across several model families and benchmarks, with modest measured training-time overhead.

Critical strengths

S1. The reported improvements over the matched OPDLM baseline are substantial on several benchmarks. With initialization, training data, optimization, and decoding settings controlled, CrossDLM improves Qwen3.5-4B from 85.8% to 92.5% on GSM8K and from 68.2% to 78.8% on MATH-500. These results provide meaningful evidence for the effectiveness of the overall supervision recipe, although the contribution of each component remains less clear.

S2. The measured training overhead is modest relative to the reported gains. In the 4B experiment, training time increases from 87.15 to 90.73 hours, approximately 4.1%. This suggests a favorable practical trade-off in the evaluated setting.

S3. The mathematical development is clear and internally consistent. The forward-KL decomposition motivates conditional marginal supervision, and the importance-weighting derivation justifies the estimator under exact local scoring. The paper explicitly separates these results from the practical Taylor approximation and finite-sample estimator, clarifying where the theoretical guarantees no longer apply.

Critical weaknesses

W1. The downstream contribution of importance weighting is not isolated. Although the weighting scheme is theoretically justified under exact local scoring, the experiments do not compare it with uniform aggregation using the same one-step proposals and completion budget. Such a comparison would clarify whether importance correction provides additional benefits beyond aggregating multiple local targets under the practical approximation.

Questions for authors

Q1. With K=4K=4 one-step completions and Taylor scoring fixed, how does importance-weighted aggregation compare with uniform averaging on downstream benchmarks?

Q2. Figure 8 shows a less clear advantage for Taylor scoring at full token substitution α=1\alpha=1 in the four-token settings. Does Taylor scoring nevertheless produce more accurate token probability distributions than causal-prefix scoring in these settings?

3 comments

We thank the reviewer for the careful questions about our estimator and target approximation. Below, we provide controlled ablations and direct target-quality evidence.

W1/Q1: is importance weighting essential at fixed K and budget?

The importance correction behaves as predicted at the estimator level. Using the same 512 nested proposal draws and exact component scores, at 4B/K=4 the importance-weighted estimator (beta=1) has lower total variation to the exact captured-space teacher marginal than uniform aggregation (beta=0): 0.1270 versus 0.1537, with beta0-minus-beta1 +0.0268 (95% CI [+0.0048,+0.0486]). K=1 is the exact negative control.

Downstream, the matched beta=1 and beta=0 results are broadly comparable, with importance weighting modestly better on our primary reasoning tasks. The Qwen3.5-9B K=4 runs share initialization, data, four proposals per state, and 7,727 optimizer updates:

DatasetOPDLMbeta=0beta=1beta1-beta0
GSM8K85.6091.1392.65+1.52
MATH-50070.6079.2080.20+1.00

Importantly, beta=0 is not an OPDLM objective: it retains CrossDLM's on-policy multi-anchor sampling, suffix-aware bidirectional target, and Taylor scoring, changing only the aggregation weights. The gains shared by beta=0 and beta=1 therefore reflect the full CrossDLM objective rather than importance weighting alone. Their ordering is less stable across the broader task suite, consistent with a finite-K bias--variance trade-off: beta=1 corrects proposal bias but can have noisier sequence-level weights at K=4, whereas beta=0 is biased but lower variance. Together with the target-fidelity result above, this supports importance weighting as the principled correction, while attributing CrossDLM's overall gains to the complete method.

Q2: target quality for a full token substitution

We thank the reviewer for raising the full-substitution question. To evaluate it directly, we conducted an actual-B16 analysis on 768 positions with positive visible right context. Relative to the causal-prefix approximation, the Taylor target improves restricted-support forward KL to the frozen-teacher rescore by 0.1308 (95% question-bootstrap CI [0.0433,0.2216]), Spearman rank correlation by 0.00948, and pairwise candidate-ordering agreement by 0.00335. The native Top-16 support retains 99.9829% of target probability mass. These results show that the suffix-aware correction improves full-substitution forward KL and candidate ordering under the tested B16 setting.

Thank you for your response. My concerns have been largely addressed. Could the authors clarify which benchmarks are included in the “broader task suite”, for example, whether it includes coding tasks, and provide the corresponding β=0 versus β=1 results?

I maintain my original score.

Thank you for the helpful follow-up. Yes, our broader evaluation includes coding tasks in addition to mathematical reasoning, general knowledge, and multilingual understanding. We have completed the matched evaluation on the benchmarks reported below and are further expanding the suite for the revision.

These results are from the same matched Qwen3.5-9B K=4 terminal runs, with identical initialization, data, four proposals per state, and 7,727 optimizer updates. β=1 denotes our full importance-weighted method, while β=0 replaces the importance weights with uniform aggregation. General-task values are cap-aware accuracy, while coding values use official pass@1.

Benchmarkβ=0β=1
GSM8K91.1392.65
MATH-50079.2080.20
AIME-2510.0023.33
MMLU-Pro59.6559.48
CEval77.8678.08
LiveCodeBench-v617.4015.42
Codeforces1.863.45

These results show that β=1 is stronger on our primary mathematical-reasoning benchmarks, while the two variants are essentially comparable on MMLU-Pro and CEval; the coding results are mixed. This pattern is consistent with the finite-K bias–variance trade-off discussed in our response: β=1 corrects proposal bias and yields a more faithful estimate of the teacher marginal, while uniform aggregation may occasionally benefit from lower finite-sample variance on particular downstream tasks. We thank the reviewer for encouraging this clarification.

Reviewer_LyhiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework for distilling language knowledge from an autoregressive (AR) teacher into a diffusion student, and addresses the generation process mismatch between the two. Specifically, the authors argue that causal generation requires the teacher to predict the next token with prior tokens only, whereas the diffusion student can utilize later information. The prior method, OPDLM, supervises the student with the teacher's next-token prediction on the student's completed text, which ignores this. Therefore, they reformulate the distillation objective to match the teacher's probability for each masked position given all visible tokens, summing over the possible values of the other masked tokens. Since this sum is intractable, they approximate it in two steps: a gradient-based (first-order) estimate of how swapping one token changes the likelihood of the text after it, and an importance-weighted average over multiple completions sampled from the student's own one-step prediction. Using OPDLM's on-policy setup, the method beats OPDLM on most of 12 benchmarks for Qwen3.5-4B and 9B and on three other model families, with 4.1% more training time.

Critical strengths
  1. The analysis of the existing on-policy distillation method for AR to diffusion models (i.e., OPDLM) is insightful, which uncovers a critical issue in OPDLM's objective that can leak the student's own bias into the target prediction, due to mismatched generation processes between AR teacher and diffusion student.
  2. Under matched data, optimization and decoding, the gains over OPDLM are consistent across five backbones, and math reasoning improves on all of them. The training overhead is small (90.73 vs 87.15 hours at 4B).
Critical weaknesses
  1. The evaluation results lack several meaningful dimensions:

    i. The paper does not show the performance gap between the AR teacher and the diffusion student, which is important to understand the cost of the proposed self distillation method.

    ii. There is also no measure of decoding efficiency or comparison against speculative decoding, which reproduces the teacher's outputs exactly and still gives a speedup. Parallel decoding is the reason to build a DLM, so without this the paper only shows the conversion loses less than one baseline.

    iii. The one capability where a bidirectional model clearly differs from an AR model, infilling, is not evaluated either.

  2. The motivation is not well backed by the paper's experiment settings. All training and evaluation use 4-token blocks with confidence-thresholded reveals. The visible right context is therefore at most three tokens, and for an AR-initialized student it may often be empty. A natural question arises: how often a supervised position actually has visible right context. However, no statistics on either this or the reveal order are given, so it is unclear whether the gains come from bidirectional conditioning at all.

Questions for authors
  1. What do the AR teachers (Qwen3.5-4B, Qwen3.5-9B, Qwen3-4B) score under the same prompts, output limits and scorers?
  2. What parallelism does the student actually achieve at evaluation? How many tokens on average revealed per forward pass, and what about wall-clock time against the AR teacher?
  3. After distillation, does the student's reveal order tend to follow the AR left-to-right pattern? For example, how often is the revealed position the leftmost masked one in its block, at evaluation and separately on math versus other tasks, and does this differ between CrossDLM and OPDLM? Relatedly, in training states, what fraction of supervised positions have at least one visible token to their right?
  4. (Suggestion) Does the method still help with 16-token or longer blocks, where parallel decoding would matter most?
  5. Can a token be revised after it has been revealed? If not, how does the student behave when the visible right context contains an error?
  6. (Suggestion) How does the converted student compare with its teacher plus speculative decoding in accuracy and latency, and how does it perform on infilling, where bidirectional context should matter most?
  7. On Qwen3-4B, the one base model with published OPDLM results, the OPDLM run here is somewhat lower on several benchmarks (e.g., MBPP-base 43.92 vs. 57.7), and training uses a constant learning rate of 3e-7 rather than OPDLM's published schedule (1e-5 decayed to 1e-6). Was the learning rate tuned for either method, and do the gains hold when OPDLM is trained with its published recipe?
3 comments

We thank the reviewer for the thoughtful questions on source-model retention, decoding behavior, and efficiency. We address each point below with additional experiments and clarifications.

W1/Q1/Q2: source AR comparison and decoding efficiency

We thank the reviewer for emphasizing this comparison. On terminal Qwen3.5-4B, cap-aware Source AR / CrossDLM is 92.42 / 91.05 on GSM8K, 79.40 / 74.20 on MATH-500, and 74.11 / 76.31 on MMLU. Conversion is not lossless, as expected with a finite distillation corpus and a changed prediction interface, but it preserves most source quality while enabling blockwise decoding. On Qwen3-4B, CrossDLM also recovers approximately 43% of OPDLM's conversion loss on both GSM8K and MATH-500.

In a fixed-token test on the same H100/PyTorch-SDPA/BF16 stack (128-token prompt and output, batch 1), CrossDLM uses 16-token diffusion blocks and eight denoising forwards per block (B=16,d=8), committing about two tokens per forward. Generating 128 tokens therefore requires eight blocks and 64 forwards, taking 2.991 s. Raw AR uses 128 decode steps and takes 5.571 s without KV cache or 4.878 s with the standard Hugging Face cache: 1.86x/1.63x speedups, respectively, and 42.79 versus 22.97/26.24 tokens/s. Across five batch/length settings, speedups are 1.86--1.95x versus cache-free AR and 1.58--1.78x versus cached AR. This isolates sequential depth, not production serving. On our highly optimized Qwen3.5 stack using SGLang, raw AR remains 27.7% faster (4.240 versus 5.863 s), reflecting more mature AR systems such as vLLM/SGLang; DLM-specific optimization, as explored by work like SparseD, Step-dLLM and FLARE, is complementary future work.

Q3: how often is visible right context available?

We thank the reviewer for prompting this analysis. Across all 541 IFEval prompts, roughly two in five blocks contain a right-before-left reveal: 37.03% for CrossDLM and 43.17% for OPDLM. The rates are comparable in magnitude, with CrossDLM modestly more left-to-right. The key distinction is supervision: CrossDLM uses the available right context in its target, whereas OPDLM retains a causal-prefix target. Position-level and GSM8K statistics are in the appendix.

Q4: do the gains survive larger blocks?

To test this directly, we trained two matched B16 OPDLM/CrossDLM pairs (seeds 42 and 10086), rather than decoding B4 checkpoints with larger blocks:

SeedDatasetOPDLMCrossDLMGain
42GSM8K71.2779.45+8.19
42MATH-50050.8062.60+11.80
10086GSM8K72.1083.17+11.07
10086MATH-50048.6063.40+14.80

Both seeds therefore show gains on both reasoning tasks.

Q5: can an incorrect revealed token be revised?

No. Our current implementation follows the standard DLM sampling rule, which updates only MASK positions, so a committed token is not revisited. This is a property of the sampler, not of CrossDLM: as a general training framework, CrossDLM is compatible with revision-capable samplers such as confidence-based remasking, which we leave as a natural extension.

Q6: relation to speculative decoding

We thank the reviewer for raising this comparison. Speculative decoding and CrossDLM are orthogonal: the former uses a draft model to accelerate an AR target while preserving its distribution, whereas CrossDLM trains the student for blockwise generation. CrossDLM can itself serve as the draft in an exact speculative-decoding pipeline, making its combination with AR verification a promising future direction.

Q7: comparison under OPDLM's published recipe

We thank the reviewer for raising recipe alignment. The original experiments did not tune the objectives separately: both used the same stable default LR (3e-7), not an objective-specific optimum. We therefore retrained both methods under OPDLM's recipe. Because its launcher and paper specify different curricula (100-outer cosine versus 10-iteration linear), we evaluate both interpretations with all other settings matched. K=2 applies only to CrossDLM, not OPDLM.

ControlMetricOPDLMCrossDLMGain
Released-code B4/d4GSM8K85.3787.95+2.58
Released-code B4/d4MBPP-base51.3256.88+5.56
Literal-paper B4/d4GSM8K85.6087.72+2.12
Literal-paper B4/d4MBPP-base51.8562.43+10.58
Literal-paper B16/d16GSM8K68.9282.26+13.34

Literal-paper B4 gain CIs are [+0.30,+3.94] on GSM8K (p=.0251) and [+5.56,+15.61] on MBPP-base (p=1.10e-4); B16 GSM8K is [+10.61,+16.07] (p=1.10e-21). As an evaluator check, the released OPDLM-4B checkpoint obtains 58.20% on MBPP-base, close to the paper's 57.7%. Table values are fresh retraining results under matched hardware and training/evaluation setups, and terminal checkpoints were fixed before evaluation.

Thank you for the detailed response and the new experiments. My main concerns have been adequately addressed by the additional results, including the comparison with the AR teacher, the decoding efficiency measurement, the right-context statistics, the B16 experiments and the retraining under OPDLM’s published recipe. I will keep my rating at 3, and encourage the authors to include these results in the final version.

Thank you for your thoughtful consideration and constructive feedback! We appreciate your suggestions and will incorporate the additional results into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.