- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DIAL, a block-size-elastic diffusion language model that supports different block sizes at inference time using one set of parameters. The main idea is mixed block-size training (MBT), where the training block size is sampled from {1, 2, 4, 8, 16}. The method further introduces per-block noise conditioning, recurrent draft refinement, a two-stream cached architecture, and argmax-based early stopping. Experiments show that one DIAL checkpoint maintains relatively stable perplexity across block sizes, while fixed-block BD3-LM models degrade when evaluated away from their training block size.
Critical strengths
-
The paper studies a useful deployment problem: supporting multiple inference block sizes with a single checkpoint. The full train/evaluation grid gives clear evidence that fixed-block models degrade off-design, while DIAL remains substantially more stable.
-
The empirical analysis is careful. The ablations help separate the role of MBT from the additional architectural components, and the paper is unusually explicit about limitations and negative results. In particular, the benefit of recurrent draft refinement grows with block size in the expected direction.
Critical weaknesses
-
The novelty relative to concurrent Adaptive Block Diffusion (ABD; Jain, 2026) is not sufficiently clear to me. Both methods randomize block configurations during training and aim to make one model work across decoding scales. The manuscript states that ABD was evaluated at a single operating point, but the public ABD manuscript appears to make a broader across-scale claim. The extra DIAL components and more extensive evaluation may still constitute a useful contribution, but the current novelty distinction seems overstated.
-
The main quality-compute trade-off is not fully validated for the actual M4-enabled sampler. Early stopping changes the generative distribution because remaining masked positions are committed using the final argmax, while the reported perplexity is sampling-free and therefore does not measure this change. In addition, most specialists in the main comparison do not use M1-M3, so the main grid is not fully architecture-matched.
Questions for authors
-
Could the authors clarify the distinction from ABD, especially regarding the claim that ABD only evaluates a single operating point? More broadly, the related-work discussion on adaptive inference in diffusion LMs could include recent methods operating on complementary axes, such as DAEDAL (arXiv:2508.00819), Prophet (arXiv:2508.19982), and Sequential Diffusion Language Models (arXiv:2509.24007). A clearer positioning relative to this line of work would materially affect my novelty assessment.
-
Can the authors evaluate the quality of the actual M4-enabled generation process against M4-disabled decoding, preferably at matched compute or latency? Since M4 changes the sampling distribution, a generation-based comparison would materially affect my confidence in the claimed quality-compute frontier. Architecture-matched fixed-block controls at additional block sizes would also help separate elasticity from the gains of the other modules.
Response to Reviewer EWUU
We thank Reviewer EWUU for the positive assessment of the problem formulation and experimental analysis. We especially appreciate the comments on the novelty distinction from ABD and the need to validate the actual generation behavior of M4.
Comment 1: Novelty relative to ABD and positioning among adaptive inference methods
We agree that our original distinction from ABD was not sufficiently precise. Both ABD and DIAL train a shared model across multiple block sizes, and ABD does report results across multiple block sizes. We have corrected the related discussion and no longer characterize ABD as evaluating only a single block-size operating point.
Our distinction is instead in the training and architectural approach. ABD uses a two-stage procedure consisting of MDLM pretraining followed by adaptive block-size fine-tuning, whereas DIAL trains from scratch using uniform mixed block-size sampling. DIAL introduces per-block noise conditioning and recurrent draft refinement to support this training regime, together with a two-stream attention implementation for efficient context reuse.
Table 4 reports the available benchmark comparison. We emphasize that these are reported results rather than a fully controlled reproduction. Table 5 further isolates the contribution of MBT: compared with a fixed-(L'=4) DIAL specialist retaining M1–M3, full DIAL performs better on LAMBADA and PTB, although the specialist has a small WikiText-103 advantage at its training operating point.
We have also expanded the related-work discussion to position DIAL alongside the methods suggested by the reviewer:
DAEDAL: adapts total output length.
Prophet: adapts the number of inner denoising/refinement steps.
Sequential Diffusion Language Models (SDLM): adapts prefix acceptance for block-diffusion-style generation from pretrained autoregressive models.
ABD and DIAL: address adaptation across block granularity.
This distinction makes clear that DIAL addresses block-size elasticity rather than claiming to introduce adaptive inference in diffusion language models generally.
Comment 2: Does M4 preserve actual generation quality, and are the controls architecture-matched?
We agree that this question cannot be answered using sampling-free likelihood metrics alone. M4 changes the generation process, so its effect must be evaluated on actual generated samples.
We therefore conducted the direct M4-on/M4-off generation comparison in Table 2. At (L'=8), argmax commitment reduces decoding time by approximately 18.4%, while Distinct-4 increases from 0.884 to 0.953 and Gen-PPL changes from 32.72 to 34.36. At (L'=4), the results show a more substantial trade-off, and at (L'=16), M4 rarely activates.
We also added the architecture-matched fixed-(L'=4) control in Table 5. This comparison retains M1–M3 in both models and changes whether mixed block-size training is used. It therefore helps separate the effect of MBT from the additional architectural components.
We have revised the manuscript to state the scope of these results accurately: M4 can reduce computation with a configuration-dependent quality trade-off, while MBT provides benefits beyond those of a fixed-block model using the same supporting architecture.