acceptodds
Withdrawn from ICLR 2027

Don’t Retrain—Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion language models (DLMs) have recently demonstrated capabilities that complement standard autoregressive (AR) models, particularly in non-sequential generation and bidirectional editing. Although recent work has shown that pretrained autoregressive checkpoints can be converted into diffusion language models, existing recipes primarily transfer parameters through continued denoising training with objective- and attention-level modifications. We instead ask whether the internal representation geometry learned by next-token prediction can be explicitly preserved during ARDLM conversion. We hypothesize that much of the semantic structure learned by AR pretraining can transfer across generation orders, and thus DLM training should be viewed as relearning the decoding path rather than relearning language representations. To investigate this, we introduce REPR-ALIGN, a Representation Alignment objective that adapts a bidirectional masked diffusion model to reuse representations from a pretrained AR model of identical architecture. Concretely, we align the hidden states of the DLM to the frozen AR model at every layer using cosine similarity, while optimizing the standard masked denoising objective. This simple alignment—with no adapters and no architectural changes beyond the attention mask—yields up to 4 training acceleration in our setting and is particularly effective in low-data regimes. Our results suggest that linguistic representations are universal regardless of generation order, and representation alignment could be a new go-to technique for training diffusion language models. Code will be made publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language models, autoregressive language models, representation alignment, language model adaptation, masked diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_p29uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the conversion of pretrained autoregressive models into masked diffusion language models. It proposes Repr-Align, which trains the diffusion student with the masked denoising loss plus a layer-wise cosine loss that aligns the student’s hidden states on masked inputs to those of the frozen AR teacher on clean inputs. Experiments on Qwen3 models (0.6B to 4B) show better HumanEval pass@10 than training without alignment, along with results on freezing parameter blocks and training on a small data subset.

Critical strengths
  1. The method is simple and easy to adopt. It adds no adapters or architectural changes, and the only extra cost is a forward pass of the frozen teacher.
  2. Under the authors’ setup, alignment gives consistent improvements over the unaligned baseline at both 0.6B and 1.7B, with clearly faster progress early in training.
Critical weaknesses
  1. The source of the gains is not separated from distillation. The teacher sees the clean sequence while the student sees the masked one, so at masked positions the alignment target contains information about tokens the student cannot see. This is close to feature-level distillation from the AR teacher, not only preservation of representation geometry. The only baseline is the authors’ own denoising recipe without alignment. Could the authors compare with logit-level distillation from the AR teacher at masked positions [1] (or maybe with existing AR-to-DLM conversion methods [2][3], but at least [1]) under the same data and steps? This would show whether aligning hidden states adds value beyond standard distillation.

  2. The evaluation is narrow relative to the claims. All experiments use code benchmarks with 128 generated tokens, while the paper concludes that linguistic representations are universal across generation orders and that alignment could become a standard technique for training DLMs. Could the authors add evaluations on general language, math, or reasoning tasks?

[1] https://arxiv.org/abs/1503.02531
[2] https://arxiv.org/abs/2410.17891
[3] https://arxiv.org/abs/2512.14067

Questions for authors
  1. How does Repr-Align compare with logit-level distillation from the AR teacher under the same data and training steps?
  2. Does the improvement hold on non-code tasks?
Reviewer_2BBJOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies converting pretrained autoregressive language models into masked diffusion language models. Repr-Align initializes a bidirectional student from the same checkpoint as a frozen causal teacher and adds layer-wise cosine alignment between student features from corrupted sequences and teacher features from clean sequences. The conversion recipe also includes masked denoising and PAPL losses, with PAPL shared by the baseline and aligned models. Experiments use Qwen3 checkpoints at 0.6B, 1.7B, and 4B on code-generation benchmarks. The paper reports improved conversion performance, explores parameter freezing and reduced unique training data, and interprets these results as evidence that pretrained representations can be reused across generation orders.

Critical strengths
  1. A practical and simple adaptation mechanism. Using a frozen, same-architecture AR teacher enables direct feature alignment without adapters or additional student parameters. This is a useful way to exploit an existing checkpoint, and the teacher is needed only during training. The contribution is a concrete adaptation recipe with a low implementation burden.
  2. Promising gains beyond initialization alone. Figure 3 reports HumanEval pass@10 improvements from 24.9 to 31.0 at 0.6B and from 31.1 to 40.5 at 1.7B under a matched-step comparison. The baseline already uses AR initialization and PAPL, making this a relevant comparison for the added alignment objective. Table 2 also investigates the metric, alignment weight, and layer selection. These results provide useful empirical motivation, subject to the configuration clarification below.
Critical weaknesses
  1. The evidence does not establish the headline efficiency and performance claims. The abstract reports up to 4× training acceleration, but the principal learning curves use optimization steps, while alignment requires an additional teacher forward pass. Figure 4 measures throughput across freezing configurations within aligned training; it does not establish the end-to-end cost advantage over a similarly frozen, unaligned baseline at matched quality. Likewise, Figure 5 establishes robustness to a smaller unique corpus, with replacement sampling and unchanged steps, rather than a reduction in processed tokens or compute. Its missing tiny-data, unaligned control limits attribution of this benefit to alignment. The comparative performance is also narrower than the frontier language suggests: in Table 1, oDLM-4B exceeds Dream-7B on HumanEval pass@10 (61.59 versus 59.2), but is substantially lower on HumanEval pass@1 (30.49 versus 56.7) and MBPP pass@10 (37.0 versus 56.2). These gaps do not by themselves invalidate the contribution, but make a transparent cost–quality comparison central to its claimed significance.
  2. The implemented objective and the interpretation of the gains are insufficiently resolved. Equation (2) and Section 4.1 specify alignment at all token positions and transformer-block outputs. Appendix B.4 instead specifies only masked, shift-valid positions and includes embedding and final post-normalization states. These are materially different objectives. Furthermore, the 1.7B scores 31.1/40.5 label the no-alignment/alignment comparison in Figure 3, but label different freezing configurations in Figure 4, whose caption says all runs use alignment. This does not establish a confound, but requires a clear mapping from runs to configurations before attributing the gains. Even if resolved, successful regularization of one model family on code benchmarks does not establish generation-order-independent representations. The Introduction's inference from L! possible paths to an L-fold compute requirement also lacks assumptions or a derivation; counting paths alone does not imply that training-cost relationship. The methodological ambiguities and stronger mechanistic interpretation together limit the conclusions that can currently be drawn.
Questions for authors
  1. Actual objective and comparison controls: Which token positions and hidden states were aligned in the reported runs, and how were teacher/student indices handled under the one-token shift? Please reconcile Sections 3.3/4.1 with Appendix B.4 and identify the alignment and freezing settings of each Figure 3/4 run, particularly the repeated 31.1/40.5 results. A consistent configuration mapping would substantially reduce my concern about attribution.
  2. Measured conversion cost: What specific quality threshold, baseline, and cost measure produce the stated 4× acceleration? Please provide teacher-inclusive time or GPU-hours to that threshold with the same freezing policy, and identify the hardware, processed-token count, and training checkpoint used for the 4B result. This would establish whether the observed step efficiency translates into the claimed computational advantage.
  3. Comparative evaluation: Were Table 1's external baselines rerun under Appendix B.7 or taken from earlier reports? Their values match the corresponding rows of Table 4 in Peng et al., Planner Aware Path Learning in Diffusion Language Models Training, which includes results adopted from prior work. Please clarify the provenance and relevant decoding/evaluation differences, and the uncertainty of the 2.39-point HumanEval pass@10 advantage over Dream.
  4. Low-data attribution: Is a no-alignment run on the same 0.8B-token subset available, with freezing and optimization settings matched to Figure 5? This single control would determine whether the reported low-data behavior is specific to alignment or also occurs with ordinary conversion. If unavailable, the current evidence supports the narrower claim that the aligned recipe can use a smaller unique corpus.
Reviewer_stcyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Repr-Align converts a pretrained autoregressive transformer into a masked diffusion model by switching to bidirectional attention and adding a layer-wise cosine alignment objective. The frozen AR teacher processes clean sequences, while the student processes corrupted sequences. The alignment term supplements masked denoising and a shared path-planning auxiliary loss. Experiments use Qwen3 checkpoints at several scales and code-generation benchmarks. Matched-step comparisons at 0.6B and 1.7B report improved HumanEval pass@10 with alignment. Additional experiments examine freezing embeddings/MLPs and repeatedly training on a smaller unique-data subset. A 4B converted model is compared with public diffusion models. The paper interprets these findings as evidence that preserving AR representations improves conversion efficiency.

Critical strengths

S1. A simple and potentially reusable conversion objective. Aligning same-architecture student and teacher hidden states avoids additional adapters or changes to inference architecture. The matched-step comparisons use the same initialization, denoising recipe, and path-planning loss, providing useful evidence that the alignment term improves the evaluated conversion procedure. S2. The experiments explore several practical dimensions of adaptation. The paper studies model scale, alignment metric and strength, layer selection, parameter freezing, and reduced unique-data availability. The reported HumanEval improvements at 0.6B and 1.7B suggest that representation-based supervision is worth further investigation, while selective parameter updates provide a potentially useful engineering option.

Critical weaknesses

W1. The training-efficiency and representation-universality claims exceed what the comparisons establish. Matched optimization steps are not matched compute because alignment requires an additional teacher forward pass and hidden-state processing. The paper does not provide sufficient time-to-target wall-clock, GPU-hour, or FLOP accounting to substantiate the headline “up to 4× training acceleration.” The 0.8B-token experiment reduces unique data but retains the optimization budget through repeated sampling, so it does not establish a comparable reduction in processed tokens or training compute. Moreover, performance improvements from clean-teacher supervision do not by themselves show that linguistic representations are universal across generation orders. W2. Evaluation and implementation inconsistencies limit confidence in the central comparisons. Table 1 reports MBPP+ scores above MBPP scores, which requires explanation of differing task sets, samples, prompts, or scoring protocols. Under a shared task set and shared generated programs, adding tests cannot increase pass@k. The main text also describes alignment over all positions and transformer-block outputs, whereas Appendix B.4 specifies masked, shift-valid positions and includes additional hidden states. Finally, Figure 4’s baseline score of 31.1 matches the unaligned 1.7B score in Figure 3, although its caption says all runs use alignment. These issues make the exact experimental comparisons difficult to reconstruct.

Questions for authors

Q1. How is the “up to 4×” acceleration calculated? Please report time-to-target performance, including the frozen teacher forward, alignment computation, memory overhead, and training hardware. How does the result change when compared at matched GPU-hours or FLOPs? Q2. Please report exact processed non-padding tokens and unique tokens for the full and tiny-data runs. With 200,000 updates, global batch size 96, and maximum length 4,096, the nominal maximum student token-position budget is approximately 78.6B; how does this relate to the reported 50B corpus and 0.8B subset? Q3. Can you specify dataset versions, task IDs, sample identities, prompts, and scoring settings for MBPP and MBPP+? Please evaluate the same generated programs on base and additional tests for the common task set, and clarify whether public-model results were rerun or taken from other sources. Q4. Which alignment implementation produced the main results: all positions or masked positions, and transformer-block outputs only or the full hidden-state tuple? Does Figure 4’s baseline include alignment? Please reconcile its 31.1 score with Figure 3. Q5. Can you provide repeated-seed uncertainty and a control distinguishing representation preservation from generic auxiliary supervision or regularization—for example, teacher-input corruption controls or parameter anchoring? A matched 4B conversion baseline would also help isolate alignment’s contribution to the scale-up result.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.