acceptodds
Under review as a conference paper at ICLR 2027

Implicit EOS Density for Online Bidirectional Length Adaptation in Masked Diffusion LLMs

Abstract

Masked diffusion LLMs (e.g., LLaDA) typically rely on a fixed generation length (i.e., a fixed MASK canvas), struggling to accommodate responses whose required lengths vary substantially across inputs: a short length may constrain content generation, while a long length wastes computation. We study the denoising dynamics and find that the evolving implicit density () of end-of-sequence (EOS) tokens during denoising reveals whether the current length is excessive or insufficient. Building on this observation, we propose -EOS, a training-free, online bidirectional length adaptation strategy for masked diffusion LLMs: low density triggers expansion, while high density triggers contraction. This enables online length adaptation within the original denoising loop, without retraining or a separate length-adjustment stage. Extensive experiments on mathematical reasoning and code generation tasks show that -EOS yields a more favorable Pareto frontier in the quality-efficiency trade-off. Additionally, because -EOS targets online length adaptation, it is orthogonal to pruning, caching, and other acceleration-oriented decoding strategies, making it potentially compatible with and complementary to them.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion large language model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qFnxOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Masked diffusion LLMs decode on a fixed-size MASK canvas, so the generation length must be committed upfront and often mismatches the length the response actually needs. This paper observes that during denoising, the implicit EOS density ρEOS — the fraction of remaining MASK positions whose implicit x0 prediction is EOS (Eq. 5) — tracks length sufficiency: it stays low when the canvas is too short, rises early when the canvas is too long, and settles at an intermediate equilibrium when the length is appropriate. Building on this, the authors propose ρ-EOS, a training-free controller inside the standard denoising loop: append MASKs when ρEOS < low, trim the sequence tail when ρEOS > ρhigh, and hold inside the band, with constant, linear, or exponential adjustment-factor functions setting the step size. On LLaDA-Instruct-8B and LLaDA-1.5-8B across GSM8K, MATH500, MBPP, and HumanEval, ρ-EOS started from Linit = 64 matches or approaches the accuracy of the best fixed length and of DAEDAL while using far fewer tokens and much less runtime (e.g., GSM8K: 84.2 acc in 823 s vs. 83.9 in 8238 s for fixed-1024), is robust to Linit from 128 to 1024, and stacks with acceleration methods such as DPad and Fast-dLLM.

Critical strengths

S1. The central observation is simple, well-supported, and useful: the implicit EOS density over remaining MASK positions is a free, online readout of length sufficiency in masked dLLMs, and the paper explains both why the signal exists (padding EOS tokens are masked and trained on, Appendix B.1) and why it beats EOS confidence (the EOS trap, Appendix B.2; Table 4: single-stage density 84.2 vs. confidence 73.7 on GSM8K). Turning it into a training-free, single-loop, bidirectional expand/hold/contract controller is a clean, natural design that removes DAEDAL's separate adjustment stage.

S2. The empirical coverage is unusually thorough for a training-free decoding paper: two 8B LLaDA variants × four benchmarks with five metrics (Tables 1-2), robustness to Linit from 128 to 1024 (Table 3), ablations on thresholds, adjustment factor, contraction, and control signal (Tables 4-6, 9-10), compatibility with Fast-dLLM/Prophet/DPad (Table 8; e.g., DPad cuts GSM8K runtime from 645 s to 324 s), and preliminary transfer to summarization, long-form generation, and IFEval (Table 11). The efficiency gains are large and consistent, e.g., on GSM8K with LLaDA-Instruct-8B, 84.2 acc and 70.0% Eratio in 823 s versus 83.9 acc and 27.6% Eratio in 8238 s for fixed-1024.

Critical weaknesses

W1. The quality-efficiency narrative overclaims in places. ρ-EOS does not dominate DAEDAL or the best fixed length everywhere: on HumanEval with LLaDA-Instruct-8B (Table 1) it reaches 43.9 (Asym) / 43.3 (Sym) versus DAEDAL 44.5 and fixed-2048 48.2, and on MATH500 with LLaDA-1.5-8B (Table 2) 41.0 versus DAEDAL 42.6 and fixed-1024 42.8, so the "Pareto dominates DAEDAL" phrasing in Sec. 4.3 holds only in the latency-dominated region of the frontier. Relatedly, contraction is partly bought with accuracy: disabling it gains +0.2 to +1.8 points on all four benchmarks (Table 9).

W2. Reproducibility and scope gaps. The max adjustment steps N, the confidence threshold τhigh, the remask/step schedule, and the exact symmetric threshold pair behind "Sym" in Tables 1-3 are never instantiated (the Sym column of Table 1 matches [0.2, 0.8] in Table 5, but this has to be inferred by matching numbers); Truntime lacks hardware, batch, and denoising-step details; no standard deviations are reported despite 3-run averaging; and all experiments use the LLaDA family, leaving open whether the ρEOS signal transfers to masked dLLMs whose training treats padding EOS differently.

Questions for authors

Q1. Could you specify the missing decoding configuration: N, τhigh, the denoising step schedule, the exact Sym threshold pair used in Tables 1-3, Lmax for Tables 1-3, and the hardware/batch size behind Truntime, plus per-run standard deviations for Acc?

Q2. Could you revise the Pareto-dominance claim to acknowledge the cells where DAEDAL or a long fixed canvas wins on accuracy (HumanEval in Table 1; MATH500 in Table 2), e.g., by reporting the joint accuracy-latency frontier of all methods explicitly?

Q3. Does the ρEOS signal appear in masked dLLMs outside the LLaDA family (e.g., Dream, MMaDA, Mercury)? Even one small-scale check would substantially strengthen the generality claim.

Q4. On GSM8K, contraction removes 13.83 positions per event on average, and of the already-decoded positions it removes, 3,760 out of 60,709 (about 6.2%) are non-EOS (Table 10). Is there any safeguard against truncating semantic content, and is this the cause of the accuracy drop when contraction is enabled (Table 9)?

4 comments
AuthorsOpenReview ↗

Response to Reviewer qFnx: Part 1

About W1 & Q2: Quality-efficiency and Pareto claim

We thank the reviewer for this helpful suggestion. We will revise the Pareto-related claims in Section 4.3 and in the conclusion. Specifically, we will replace broad statements such as “Pareto dominates” and “superior Pareto frontier” with the more precise characterization that our method provides a favorable quality–efficiency trade-off, with Pareto dominance restricted to the latency-dominated portion of the frontier. We will also explicitly report the joint accuracy–latency Pareto frontier across all methods.

About W2 & Q1 - Reproducibility and scope gaps (decoding configuration)

We thank the reviewer for this constructive comment. In the current manuscript, Section 4.1 (“Implementation Details”) summarizes the decoding configuration—including the confidence threshold τhigh\tau_\text{high}, block length, and denoising/remasking schedule—with the statement: “We follow the settings of DAEDAL (Li et al., 2025a) to ensure fairness and reproducibility.” To make all choices explicit and reproducible, we will replace this statement with the configuration table below.

Original tablemax adjustment steps NLmaxL_\text{max}τhigh\tau_\text{high}Block lengthDenoising / remasking scheduleSym [ρlow\rho_\text{low}, ρhigh\rho_\text{high}]Asym [ρlow\rho_\text{low}, ρhigh\rho_\text{high}]Hardware for TruntimeT_\text{runtime}Batch size
Table 16420480.932steps=gen_length[0.2, 0.8][0.4, 0.8]8 × NVIDIA H2008
Table 26420480.932steps=gen_length[0.2, 0.8][0.4, 0.8]8 × NVIDIA H2008
Table 36420480.932steps=gen_lengthN/A[0.4, 0.8]8 × NVIDIA H2008

As the request of the reviewer, to quantify run-to-run variation, we additionally report results over three runs. The following values are reported as mean ± standard deviation.

Original tableBenchmarkρ\rho-EOS Sym [ρlow=0.2\rho_\text{low}=0.2,ρhigh=0.8\rho_\text{high}=0.8]ρ\rho-EOS Asym [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
Table 1GSM8K81.8 ± 0.0184.2 ± 0.09
Table 1MATH-50037.0 ± 0.0540.6 ± 0.12
Table 1MBPP39.6 ± 0.0440.6 ± 0.10
Table 1HumanEval43.3 ± 0.0943.9 ± 0.17
Table 2GSM8K83.3 ± 0.1284.5 ± 0.04
Table 2MATH-50038.2 ± 0.0741.0 ± 0.05
Table 2MBPP40.8 ± 0.1040.8 ± 0.09
Table 2HumanEval44.5 ± 0.0644.5 ± 0.09
AuthorsOpenReview ↗

Response to Reviewer qFnx: Part 2

About Q3: Scope beyond LLaDA

We thank the reviewer for this constructive suggestion. We conducted additional experiments on Dream, MMaDA, LLaDA-2.0-mini, and SDAR-8B-Chat-b32. Except Mercury (Inception Lab), cause Mercury is a closed-source model. For Dream and MMaDA, we followed the same experimental setup as in Tables 1 and 2.

Dream-v0-Instruct-7B:

BenchmarkMetric6412825651210242048DAEDAL Linit=64L_\text{init}=64ρ\rho-EOS Linit=64L_\text{init}=64 [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑39.565.181.583.983.784.583.984.5
EtokenE_\text{token}62.8122.5196.7209.7211.1215.5183.0174.7
NtokenN_\text{token}6412825651210242048239.8194.8
EratioE_\text{ratio} ↑ (%)98.195.776.841.020.610.576.389.7
TruntimeT_\text{runtime} ↓ (s)493745141521013636796518021177
MATH-500Acc. ↑26.234.441.846.446.044.845.244.8
EtokenE_\text{token}62.4121.0222.7329.0402.4590.0260.0221.8
NtokenN_\text{token}6412825651210242048333.7241.8
EratioE_\text{ratio} ↑ (%)97.594.687.064.339.328.877.991.8
TruntimeT_\text{runtime} ↓ (s)1763306351160202544001036645
MBPPAcc. ↑6.640.049.448.450.651.449.652.2
EtokenE_\text{token}58.5114.3168.2191.3199.0247.6159.7170.7
NtokenN_\text{token}6412825651210242048288.8226.0
EratioE_\text{ratio} ↑ (%)91.489.365.737.419.412.155.375.5
TruntimeT_\text{runtime} ↓ (s)2544497171032170436171381.5916.5
HumanEvalAcc. ↑9.232.356.153.751.855.554.955.5
EtokenE_\text{token}56.9121.1185.9241.8254.1268.4185.5180.7
NtokenN_\text{token}6412825651210242048310.7235.9
EratioE_\text{ratio} ↑ (%)88.994.672.647.224.813.159.776.6
TruntimeT_\text{runtime} ↓ (s)921622624316631340419268

MMaDA:

BenchmarkMetric6412825651210242048DAEDAL Linit=64L_\text{init}=64ρ\rho-EOS Linit=64L_\text{init}=64 [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑43.062.567.468.070.271.270.870.9
EtokenE_\text{token}63.5115.1178.1225.2300.4433.5130.9151.2
NtokenN_\text{token}6412825651210242048171.7179.7
EratioE_\text{ratio} ↑ (%)99.390.069.644.029.321.276.284.1
TruntimeT_\text{runtime} ↓ (s)2986271347284858621505612771061
MATH-500Acc. ↑20.925.134.634.835.035.434.635.0
EtokenE_\text{token}58.7106.4187.8296.1440.5647.4130.8123.1
NtokenN_\text{token}6412825651210242048181.9145.3
EratioE_\text{ratio} ↑ (%)91.783.173.457.843.031.671.984.7
TruntimeT_\text{runtime} ↓ (s)20537478017954375103611074643
AuthorsOpenReview ↗

Response to Reviewer qFnx: Part 3

For the block-diffusion models, LLaDA-2.0-mini and SDAR-8B-Chat-b32, we apply ρ\rho-EOS at the block level. Both models have an inherent block size of 32. With ρ\rho-EOS, the canvas size of each block is no longer fixed at 32; instead, it is adaptively adjusted according to the implicit EOS density.

LLaDA-2.0-mini:

BenchmarkMetricBaselineρ\rho-EOS [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑92.389.2
NtokenN_\text{token}330.6306.4
TruntimeT_\text{runtime} ↓ (s)65035704
MATH-500Acc. ↑69.869.8
NtokenN_\text{token}946.0934.8
TruntimeT_\text{runtime} ↓ (s)68945250
MBPPAcc. ↑69.268.8
NtokenN_\text{token}743.0708.9
TruntimeT_\text{runtime} ↓ (s)70074988
HumanEvalAcc. ↑80.580.7
NtokenN_\text{token}667.7713.5
TruntimeT_\text{runtime} ↓ (s)17911388

SDAR-8B-Chat-b32:

BenchmarkMetricBaselineρ\rho-EOS [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑87.788.9
NtokenN_\text{token}228.7229.3
TruntimeT_\text{runtime} ↓ (s)873662
MATH-500Acc. ↑59.859.8
NtokenN_\text{token}500.6496.2
TruntimeT_\text{runtime} ↓ (s)738658
MBPPAcc. ↑44.843.0
NtokenN_\text{token}148.8134.3
TruntimeT_\text{runtime} ↓ (s)302225
HumanEvalAcc. ↑68.368.0
NtokenN_\text{token}219.4187.9
TruntimeT_\text{runtime} ↓ (s)11799

Overall, these results demonstrate that ρ\rho-EOS generalizes to a broad range of masked dLLMs, including LLaDA, Dream, and MMaDA. For the block-diffusion models (LLaDA-2.0 and SDAR), their semi-autoregressive generation process means that they are not subject to the same fixed-canvas limitation. Consequently, rhorho-EOS does not consistently improve task accuracy for these models. Nevertheless, its ability to contract the canvas reduces the number of generated tokens in many settings and yields consistent runtime reductions across the reported block-diffusion experiments.

AuthorsOpenReview ↗

Response to Reviewer qFnx: Part 4

About Q4: Contraction safeguard

We thank the reviewer for this constructive suggestion. Based on the failure mode observed in Table 10, we introduce two safeguards to prevent the inadvertent truncation of semantically meaningful content. The first, safeguard-non-EOS, halts contraction upon encountering the first non-EOS token when scanning from the end of the canvas; it may therefore remove only trailing [EOS] or [MASK] tokens. The second, safeguard-committed, permits the removal of trailing [MASK] tokens only, ensuring that no committed token, including an EOS token, can be truncated. The latter will be the conservative default in the revised method, whereas safeguard-non-EOS is reported as a more aggressive optional efficiency-oriented variant.

We evaluate both safeguards under the same settings as in Table 9. The results are shown below.

ContractionGSM8kMATH-500MBPPHumaneval
Acc.remove generated token event ratioAcc.remove generated token event ratioAcc.remove generated token event ratioAcc.remove generated token event ratio
Enabled81.850.23%37.023.60%39.610.60%43.36.71%
Disabled83.10%38.00%39.80%45.10%
safeguard-non-EOS83.20%38.00%39.70%45.10%
safeguard-committed83.20%38.20%39.80%45.10%

As shown above, unconstrained contraction (“Enabled”) yields the lowest accuracy on all four benchmarks. Both safeguards reduce the generated-token-removal event ratio to zero, matching the result of disabling contraction. In particular, safeguard-committed matches or improves upon the accuracy of disabled contraction across all four benchmarks. safeguard-non-EOS is similarly comparable, with only a 0.1-point decrease on MBPP and a 0.2-point decrease on MATH-500. These results provide evidence that the proposed safeguards effectively eliminate undesirable token-removal events while preserving task performance.

Reviewer_k3R5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ρ\rho-EOS, a training-free method that adapts the generation length of masked diffusion LLMs during denoising. At each step, it computes the implicit EOS density, the fraction of remaining masked positions whose current prediction is EOS. A low density triggers expansion by appending masks, a high density triggers contraction by removing trailing positions, and an intermediate density leaves the length unchanged. Unlike DAEDAL, which only expands the canvas and relies on a separate initial length-adjustment stage, ρ\rho-EOS adjusts the length in both directions within a single denoising loop. Experiments use LLaDA-Instruct-8B and LLaDA-1.5-8B on GSM8K, MATH500, MBPP, and HumanEval. Starting from a 64-token canvas, ρ\rho-EOS reaches accuracy comparable to DAEDAL and the best fixed-length baselines, with lower runtime than DAEDAL. It also remains stable across initial lengths from 128 to 1024.

Critical strengths
  1. The implicit EOS density is a simple and practical signal. It comes from predictions the model already makes at each step, so the method needs neither training nor extra forward passes. The comparison with EOS confidence in Table 4 is convincing: when length adjustment is interleaved with denoising, confidence breaks down because of early EOS overconfidence, while density keeps accuracy close to the two-stage setting.

  2. Bidirectional control makes the method much less sensitive to the initial canvas. When the initial length is too long, DAEDAL cannot shrink it and its cost grows quickly, whereas ρ\rho-EOS contracts toward a task-appropriate length, keeping accuracy stable and runtime far lower (Table 3). In practice, this removes the need to tune the canvas length for each task.

Critical weaknesses
  1. The paper motivates ρ\rho-EOS by the fixed-length limitation of masked dLLMs, but recent dLLMs increasingly adopt block diffusion, which generates block by block and stops at EOS, so the length is variable by construction. This includes BD3-LM (Arriola et al., 2025) and SDAR (Cheng et al., 2025), which the paper cites, and LLaDA2.0 (Bie et al., 2025), which converts the LLaDA family itself to block diffusion for inference. The problem is real for the two LLaDA-8B models evaluated here, but the paper only briefly mentions block diffusion in the related work and does not explain why the fixed-canvas setting still matters. A simple training-free baseline is also missing. Although LLaDA is trained on full sequences, the fixed canvas in this paper is filled from left to right in blocks of 32 tokens (Fig. 1), so one could start from a short canvas and append another block of masks whenever the filled blocks contain no EOS. This may not work if the model’s predictions depend on the total canvas length, but without this comparison it is unclear how much the density-based adjustment adds over such simple extension.

  2. The efficiency claims are hard to verify. The decoding configuration of the fixed-length baselines is not reported, and Fig. 1 suggests they decode one token per step, while Algorithm 1 commits every position above a confidence threshold, so part of the runtime gap may come from the unmasking rule rather than from length adaptation. The runtimes are also inconsistent across tables: on HumanEval, the fixed-length baselines are several times faster in Table 2 than in Table 1, although both models share the same architecture, and with the Table 2 numbers, a fixed length of 512 is both more accurate and faster than ρ\rho-EOS.

Minor: citations are written in textual form throughout (e.g., “Large language models Brown et al. (2020); Bai et al. (2023); ... have exhibited ...”), which makes the text hard to read. Parenthetical citations would improve readability.

References

Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models. In International Conference on Learning Representations (ICLR), 2025.

Tiwei Bie, Maosong Cao, Kun Chen, et al. LLaDA2.0: Scaling Up Diffusion Language Models to 100B. arXiv preprint arXiv:2512.15745, 2025.

Shuang Cheng, Yihan Bian, Dawei Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, et al. SDAR: A Synergistic Diffusion-Autoregression Paradigm for Scalable Sequence Generation. arXiv preprint arXiv:2510.06303, 2025.

Questions for authors
  1. Given that block diffusion models such as SDAR and LLaDA2.0 support variable-length generation natively, in which settings does length adaptation on a fixed canvas remain important, and does ρ\rho-EOS bring any benefit to block diffusion models?

  2. Could the authors compare ρ\rho-EOS with a training-free blockwise extension baseline on LLaDA, which starts from a short canvas and appends a new block of masks until EOS appears, under the same block length and unmasking rule? If this baseline fails, for example because the model’s predictions depend on the total canvas length, showing this would directly support the motivation.

  3. Which decoding configuration (number of steps, block length, and unmasking rule) do the fixed-length baselines and ρ\rho-EOS use, and how are blocks handled when the canvas is extended or truncated? Could the authors add fixed-length baselines with the same threshold-based unmasking as ρ\rho-EOS, and explain why the HumanEval runtimes of the fixed-length baselines differ so much between Tables 1 and 2?

I would be glad to raise my score if ρ\rho-EOS clearly outperforms the blockwise extension baseline and keeps its efficiency advantage over fixed-length decoding under a matched unmasking rule.

2 comments
AuthorsOpenReview ↗

Response to Reviewer k3R5: Part 1

About W1 & Q2 - Training-free blockwise extension baseline

We thank the reviewer for this constructive suggestion. We compare ρ\rho-EOS with the training-free blockwise-extension baseline on LLaDA and LLaDA-1.5. The baseline starts from a short canvas and adds a new mask block with the same block length and unmask rule until EOS appears.

LLaDA:

BenchmarkMetricBlockwise Extensionρ\rho-EOS[ρlow=0.2\rho_\text{low}=0.2,ρhigh=0.8\rho_\text{high}=0.8]ρ\rho-EOS[ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc.↑55.881.884.2
EtokenE_\text{token}97.9198.4252.5
NtokenN_\text{token}98.9231.8361.0
EratioE_\text{ratio}↑(%)99.085.670.0
TruntimeT_\text{runtime}↓(s)250645823
MATH-500Acc.↑24.437.040.6
EtokenE_\text{token}141.6428.8558.7
NtokenN_\text{token}142.6486.3682.2
EratioE_\text{ratio}↑(%)99.388.281.9
TruntimeT_\text{runtime}↓(s)32913981968
MBPPAcc.↑37.439.640.6
EtokenE_\text{token}105.8324.0335.9
NtokenN_\text{token}106.8507.4594.4
EratioE_\text{ratio}↑(%)99.163.955.5
TruntimeT_\text{runtime}↓(s)50120312030
HumanEvalAcc.↑33.643.343.9
EtokenE_\text{token}164.2570.4643.1
NtokenN_\text{token}165.7654.0769.9
EratioE_\text{ratio}↑(%)99.188.483.5
TruntimeT_\text{runtime}↓(s)168580593

LLaDA-1.5:

BenchmarkMetricBlockwise Extensionρ\rho-EOS[ρlow=0.2\rho_\text{low}=0.2,ρhigh=0.8\rho_\text{high}=0.8]ρ\rho-EOS[ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc.↑56.283.384.5
EtokenE_\text{token}92.2214.5231.6
NtokenN_\text{token}93.2254.3362.4
EratioE_\text{ratio}↑(%)98.984.463.9
TruntimeT_\text{runtime}↓(s)247721990
MATH-500Acc.↑27.038.241.0
EtokenE_\text{token}138.4339.5583.4
NtokenN_\text{token}139.4373.1711.3
EratioE_\text{ratio}↑(%)99.391.082.0
TruntimeT_\text{runtime}↓(s)371850637
MBPPAcc.↑38.040.840.8
EtokenE_\text{token}119.3336.1334.1
NtokenN_\text{token}120.3535.6540.8
EratioE_\text{ratio}↑(%)99.262.861.8
TruntimeT_\text{runtime}↓(s)61330012565
HumanEvalAcc.↑35.444.544.5
EtokenE_\text{token}188.8460.8686.6
NtokenN_\text{token}189.8509.7809.5
EratioE_\text{ratio}↑(%)99.590.484.8
TruntimeT_\text{runtime}↓(s)184433647

The results show the blockwise-extension incurs slight accuracy drops on MBPP and substantial accuracy losses on GSM8K, MATH-500, and HumanEval. Although it achieves high EratioE_\text{ratio}, its accuracy degradation indicates that naively appending fixed-size blocks is not an effective approach. These results support our motivation for using the online ρ\rho-EOS signal and demonstrate the empirical advantage of ρ\rho-EOS over blockwise extension.

About Q1 - Scope relative to native block diffusion

For the block-diffusion models, LLaDA-2.0-mini and SDAR-8B-Chat-b32, we apply ρ\rho-EOS at the block level. Both models have an inherent block size of 32. With ρ\rho-EOS, the canvas size of each block is no longer fixed at 32; instead, it is adaptively adjusted according to the implicit EOS density.

LLaDA-2.0-mini:

BenchmarkMetricBaselineρ\rho-EOS [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑92.389.2
NtokenN_\text{token}330.6306.4
TruntimeT_\text{runtime} ↓ (s)65035704
MATH-500Acc. ↑69.869.8
NtokenN_\text{token}946.0934.8
TruntimeT_\text{runtime} ↓ (s)68945250
MBPPAcc. ↑69.268.8
NtokenN_\text{token}743.0708.9
TruntimeT_\text{runtime} ↓ (s)70074988
HumanEvalAcc. ↑80.580.7
NtokenN_\text{token}667.7713.5
TruntimeT_\text{runtime} ↓ (s)17911388

SDAR-8B-Chat-b32:

BenchmarkMetricBaselineρ\rho-EOS [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑87.788.9
NtokenN_\text{token}228.7229.3
TruntimeT_\text{runtime} ↓ (s)873662
MATH-500Acc. ↑59.859.8
NtokenN_\text{token}500.6496.2
TruntimeT_\text{runtime} ↓ (s)738658
MBPPAcc. ↑44.843.0
NtokenN_\text{token}148.8134.3
TruntimeT_\text{runtime} ↓ (s)302225
HumanEvalAcc. ↑68.368.0
NtokenN_\text{token}219.4187.9
TruntimeT_\text{runtime} ↓ (s)11799

We supplement experiments related to Masked dLLM such as Dream and MMaDA, as mentioned in Response to Reviewer kCj3: Part 2, these results demonstrate that ρ\rho-EOS generalizes to a broad range of masked dLLMs, including LLaDA, Dream, and MMaDA. For the block-diffusion models (LLaDA-2.0 and SDAR), their semi-autoregressive generation process means that they are not subject to the same fixed-canvas limitation. Consequently, ρ\rho-EOS does not consistently improve task accuracy for these models. Nevertheless, its ability to contract the canvas reduces the number of generated tokens in many settings and yields consistent runtime reductions across the reported block-diffusion experiments.

AuthorsOpenReview ↗

Response to Reviewer k3R5: Part 2

About W2 and Q3 - Decoding configuration and Table 1&2 humaneval timing audit

In the current manuscript, Section 4.1 (“Implementation Details”) summarizes the decoding configuration—including the confidence threshold τhigh\tau_\text{high}, block length, and denoising/remasking schedule—with the statement: “We follow the settings of DAEDAL (Li et al., 2025a) to ensure fairness and reproducibility.” To make all choices explicit and reproducible, we will replace this statement with the configuration table below.

Original tablemax adjustment steps NLmaxL_\text{max}τhigh\tau_\text{high}Block lengthDenoising / remasking scheduleSym [ρlow\rho_\text{low}, ρhigh\rho_\text{high}]Asym [ρlow\rho_\text{low}, ρhigh\rho_\text{high}]Hardware for TruntimeT_\text{runtime}Batch size
Table 16420480.932steps=gen_length[0.2, 0.8][0.4, 0.8]8 × NVIDIA H2008
Table 26420480.932steps=gen_length[0.2, 0.8][0.4, 0.8]8 × NVIDIA H2008

Humaneval timing audit:

Thank you for bringing up this point. We audited the experimental logs and found that the large discrepancy between the HumanEval runtimes of the fixed-length baselines in Tables 1 and 2 was caused by inconsistent hardware allocation. The LLaDA experiments were run on a single node with 8 NVIDIA H200 GPUs, whereas the LLaDA-1.5 experiments were inadvertently submitted on four such nodes. This additional parallelism substantially reduced the reported wall-clock time, making the original runtimes not directly comparable.

To correct this issue, we reran the Table 2 HumanEval fixed-length baselines on a single node with 8 NVIDIA H200 GPUs. The revised results are as follows.

TruntimeT_\text{runtime} on HumanEval (s)LLaDALLaDA-1.5
64111101
128230217
256543558
51214741472
102445695320
20481604615893

About the Minor - Readability of citations

We thank the reviewer for this suggestion. We will revise the manuscript.

Reviewer_kCj3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper extends DAEDAL. It is a training-free method to adjust the canvas length of a dLLM during decoding. It uses a global EOS signal from the masked positions. It can also shorten the canvas.

Critical strengths
  1. It extends DAEDAL in a natural way. The global EOS signal makes sense to me.
  2. It can also shrink the canvas. DAEDAL can only make it longer.
Critical weaknesses
  1. It feels incremental over DAEDAL. Both methods use an EOS signal to decide the length. And The accuracy gain over DAEDAL is small.
  2. “Bidirectional” is a bit overclaimed. It only pads or cuts at the right end. It does not allow changes on the left.
  3. It is only tested on the LLaDA family. I cannot tell if it generalizes to other dLLMs.
  4. the related work stops at 2025.
Questions for authors
  1. Will the code be released?
5 comments
AuthorsOpenReview ↗

Response to Reviewer kCj3: Part 1

About Weakness 1 - Novelty and relation to DAEDAL

Our central contribution is distinct: to the best of our knowledge, our work are the first to formulate implicit EOS density as an online signal of length sufficiency for dLLMs. As shown in Table 4, when length adjustment is interleaved with denoising, implicit EOS density outperforms EOS confidence. We further employ this signal into a single-stage decoding with an online expand/hold/contract controller, eliminating the need for a separate length-adjustment stage. In contrast, DAEDAL relies on trailing EOS confidence in a distinct length-adjustment stage followed by generation, and supports expansion only.

As Reviewer k3R5 noted under “Critical Strengths 1,” these distinctions constitute key improvements of our approach over DAEDAL.

The single-stage design is also operationally important. A separate pre-generation length-estimation stage increases inference latency and hinders seamless integration with existing decoding-acceleration methods for dLLMs. ρ\rho-EOS reuses predictions already computed along the denoising trajectory, introduces no additional model forward pass for length estimation, and therefore integrates directly with such acceleration methods. Table 8 in the appendix demonstrates this compatibility with the acceleration methods evaluated in our paper.

About Weakness 2 - Meaning of "bidirectional"

Here, “direction” refers to the direction of the canvas-size update—contraction versus expansion—rather than to inserting or deleting [MASK] tokens at arbitrary positions on the left or right. Our LLaDA implementation uses block decoding, which resolves left blocks before right blocks. Inserting or deleting tokens on the left disrupts this decoding order, and the substantial performance degradation we observe indicates that such operations should be avoided.

Thank you for highlighting this potential ambiguity. We will clarify this point in the revision to prevent misunderstandings for readers.

AuthorsOpenReview ↗

Response to Reviewer kCj3: Part 2

About Weakness 3 - Generality

We thank the reviewer for this constructive suggestion. We conducted additional experiments on Dream, MMaDA, LLaDA-2.0-mini, and SDAR-8B-Chat-b32. For Dream and MMaDA, we followed the same experimental setup as in Tables 1 and 2.

Dream-v0-Instruct-7B:

BenchmarkMetric6412825651210242048DAEDAL Linit=64L_\text{init}=64ρ\rho-EOS Linit=64L_\text{init}=64 [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑39.565.181.583.983.784.583.984.5
EtokenE_\text{token}62.8122.5196.7209.7211.1215.5183.0174.7
NtokenN_\text{token}6412825651210242048239.8194.8
EratioE_\text{ratio} ↑ (%)98.195.776.841.020.610.576.389.7
TruntimeT_\text{runtime} ↓ (s)493745141521013636796518021177
MATH-500Acc. ↑26.234.441.846.446.044.845.244.8
EtokenE_\text{token}62.4121.0222.7329.0402.4590.0260.0221.8
NtokenN_\text{token}6412825651210242048333.7241.8
EratioE_\text{ratio} ↑ (%)97.594.687.064.339.328.877.991.8
TruntimeT_\text{runtime} ↓ (s)1763306351160202544001036645
MBPPAcc. ↑6.640.049.448.450.651.449.652.2
EtokenE_\text{token}58.5114.3168.2191.3199.0247.6159.7170.7
NtokenN_\text{token}6412825651210242048288.8226.0
EratioE_\text{ratio} ↑ (%)91.489.365.737.419.412.155.375.5
TruntimeT_\text{runtime} ↓ (s)2544497171032170436171381.5916.5
HumanEvalAcc. ↑9.232.356.153.751.855.554.955.5
EtokenE_\text{token}56.9121.1185.9241.8254.1268.4185.5180.7
NtokenN_\text{token}6412825651210242048310.7235.9
EratioE_\text{ratio} ↑ (%)88.994.672.647.224.813.159.776.6
TruntimeT_\text{runtime} ↓ (s)921622624316631340419268

MMaDA:

BenchmarkMetric6412825651210242048DAEDAL Linit=64L_\text{init}=64ρ\rho-EOS Linit=64L_\text{init}=64 [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑43.062.567.468.070.271.270.870.9
EtokenE_\text{token}63.5115.1178.1225.2300.4433.5130.9151.2
NtokenN_\text{token}6412825651210242048171.7179.7
EratioE_\text{ratio} ↑ (%)99.390.069.644.029.321.276.284.1
TruntimeT_\text{runtime} ↓ (s)2986271347284858621505612771061
MATH-500Acc. ↑20.925.134.634.835.035.434.635.0
EtokenE_\text{token}58.7106.4187.8296.1440.5647.4130.8123.1
NtokenN_\text{token}6412825651210242048181.9145.3
EratioE_\text{ratio} ↑ (%)91.783.173.457.843.031.671.984.7
TruntimeT_\text{runtime} ↓ (s)20537478017954375103611074643
AuthorsOpenReview ↗

Response to Reviewer kCj3: Part 3

For the block-diffusion models, LLaDA-2.0-mini and SDAR-8B-Chat-b32, we apply ρ\rho-EOS at the block level. Both models have an inherent block size of 32. With ρ\rho-EOS, the canvas size of each block is no longer fixed at 32; instead, it is adaptively adjusted according to the implicit EOS density.

LLaDA-2.0-mini:

BenchmarkMetricBaselineρ\rho-EOS [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑92.389.2
NtokenN_\text{token}330.6306.4
TruntimeT_\text{runtime} ↓ (s)65035704
MATH-500Acc. ↑69.869.8
NtokenN_\text{token}946.0934.8
TruntimeT_\text{runtime} ↓ (s)68945250
MBPPAcc. ↑69.268.8
NtokenN_\text{token}743.0708.9
TruntimeT_\text{runtime} ↓ (s)70074988
HumanEvalAcc. ↑80.580.7
NtokenN_\text{token}667.7713.5
TruntimeT_\text{runtime} ↓ (s)17911388

SDAR-8B-Chat-b32:

BenchmarkMetricBaselineρ\rho-EOS [ρlow=0.4\rho_\text{low}=0.4,ρhigh=0.8\rho_\text{high}=0.8]
GSM8KAcc. ↑87.788.9
NtokenN_\text{token}228.7229.3
TruntimeT_\text{runtime} ↓ (s)873662
MATH-500Acc. ↑59.859.8
NtokenN_\text{token}500.6496.2
TruntimeT_\text{runtime} ↓ (s)738658
MBPPAcc. ↑44.843.0
NtokenN_\text{token}148.8134.3
TruntimeT_\text{runtime} ↓ (s)302225
HumanEvalAcc. ↑68.368.0
NtokenN_\text{token}219.4187.9
TruntimeT_\text{runtime} ↓ (s)11799

Overall, these results demonstrate that ρ\rho-EOS generalizes to a broad range of masked dLLMs, including LLaDA, Dream, and MMaDA. For the block-diffusion models (LLaDA-2.0 and SDAR), their semi-autoregressive generation process means that they are not subject to the same fixed-canvas limitation. Consequently, rhorho-EOS does not consistently improve task accuracy for these models. Nevertheless, its ability to contract the canvas reduces the number of generated tokens in many settings and yields consistent runtime reductions across the reported block-diffusion experiments.

About Weakness 4 and Question 1 - Related work and code

We will update related work to cover the relevant post-2025. We have released our code to the community.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.