- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Masked diffusion LLMs decode on a fixed-size MASK canvas, so the generation length must be committed upfront and often mismatches the length the response actually needs. This paper observes that during denoising, the implicit EOS density ρEOS — the fraction of remaining MASK positions whose implicit x0 prediction is EOS (Eq. 5) — tracks length sufficiency: it stays low when the canvas is too short, rises early when the canvas is too long, and settles at an intermediate equilibrium when the length is appropriate. Building on this, the authors propose ρ-EOS, a training-free controller inside the standard denoising loop: append MASKs when ρEOS < low, trim the sequence tail when ρEOS > ρhigh, and hold inside the band, with constant, linear, or exponential adjustment-factor functions setting the step size. On LLaDA-Instruct-8B and LLaDA-1.5-8B across GSM8K, MATH500, MBPP, and HumanEval, ρ-EOS started from Linit = 64 matches or approaches the accuracy of the best fixed length and of DAEDAL while using far fewer tokens and much less runtime (e.g., GSM8K: 84.2 acc in 823 s vs. 83.9 in 8238 s for fixed-1024), is robust to Linit from 128 to 1024, and stacks with acceleration methods such as DPad and Fast-dLLM.
Critical strengths
S1. The central observation is simple, well-supported, and useful: the implicit EOS density over remaining MASK positions is a free, online readout of length sufficiency in masked dLLMs, and the paper explains both why the signal exists (padding EOS tokens are masked and trained on, Appendix B.1) and why it beats EOS confidence (the EOS trap, Appendix B.2; Table 4: single-stage density 84.2 vs. confidence 73.7 on GSM8K). Turning it into a training-free, single-loop, bidirectional expand/hold/contract controller is a clean, natural design that removes DAEDAL's separate adjustment stage.
S2. The empirical coverage is unusually thorough for a training-free decoding paper: two 8B LLaDA variants × four benchmarks with five metrics (Tables 1-2), robustness to Linit from 128 to 1024 (Table 3), ablations on thresholds, adjustment factor, contraction, and control signal (Tables 4-6, 9-10), compatibility with Fast-dLLM/Prophet/DPad (Table 8; e.g., DPad cuts GSM8K runtime from 645 s to 324 s), and preliminary transfer to summarization, long-form generation, and IFEval (Table 11). The efficiency gains are large and consistent, e.g., on GSM8K with LLaDA-Instruct-8B, 84.2 acc and 70.0% Eratio in 823 s versus 83.9 acc and 27.6% Eratio in 8238 s for fixed-1024.
Critical weaknesses
W1. The quality-efficiency narrative overclaims in places. ρ-EOS does not dominate DAEDAL or the best fixed length everywhere: on HumanEval with LLaDA-Instruct-8B (Table 1) it reaches 43.9 (Asym) / 43.3 (Sym) versus DAEDAL 44.5 and fixed-2048 48.2, and on MATH500 with LLaDA-1.5-8B (Table 2) 41.0 versus DAEDAL 42.6 and fixed-1024 42.8, so the "Pareto dominates DAEDAL" phrasing in Sec. 4.3 holds only in the latency-dominated region of the frontier. Relatedly, contraction is partly bought with accuracy: disabling it gains +0.2 to +1.8 points on all four benchmarks (Table 9).
W2. Reproducibility and scope gaps. The max adjustment steps N, the confidence threshold τhigh, the remask/step schedule, and the exact symmetric threshold pair behind "Sym" in Tables 1-3 are never instantiated (the Sym column of Table 1 matches [0.2, 0.8] in Table 5, but this has to be inferred by matching numbers); Truntime lacks hardware, batch, and denoising-step details; no standard deviations are reported despite 3-run averaging; and all experiments use the LLaDA family, leaving open whether the ρEOS signal transfers to masked dLLMs whose training treats padding EOS differently.
Questions for authors
Q1. Could you specify the missing decoding configuration: N, τhigh, the denoising step schedule, the exact Sym threshold pair used in Tables 1-3, Lmax for Tables 1-3, and the hardware/batch size behind Truntime, plus per-run standard deviations for Acc?
Q2. Could you revise the Pareto-dominance claim to acknowledge the cells where DAEDAL or a long fixed canvas wins on accuracy (HumanEval in Table 1; MATH500 in Table 2), e.g., by reporting the joint accuracy-latency frontier of all methods explicitly?
Q3. Does the ρEOS signal appear in masked dLLMs outside the LLaDA family (e.g., Dream, MMaDA, Mercury)? Even one small-scale check would substantially strengthen the generality claim.
Q4. On GSM8K, contraction removes 13.83 positions per event on average, and of the already-decoded positions it removes, 3,760 out of 60,709 (about 6.2%) are non-EOS (Table 10). Is there any safeguard against truncating semantic content, and is this the cause of the accuracy drop when contraction is enabled (Table 9)?
4 comments
Response to Reviewer qFnx: Part 2
About Q3: Scope beyond LLaDA
We thank the reviewer for this constructive suggestion. We conducted additional experiments on Dream, MMaDA, LLaDA-2.0-mini, and SDAR-8B-Chat-b32. Except Mercury (Inception Lab), cause Mercury is a closed-source model. For Dream and MMaDA, we followed the same experimental setup as in Tables 1 and 2.
Dream-v0-Instruct-7B:
| Benchmark | Metric | 64 | 128 | 256 | 512 | 1024 | 2048 | DAEDAL | -EOS [,] |
|---|---|---|---|---|---|---|---|---|---|
| GSM8K | Acc. ↑ | 39.5 | 65.1 | 81.5 | 83.9 | 83.7 | 84.5 | 83.9 | 84.5 |
| 62.8 | 122.5 | 196.7 | 209.7 | 211.1 | 215.5 | 183.0 | 174.7 | ||
| 64 | 128 | 256 | 512 | 1024 | 2048 | 239.8 | 194.8 | ||
| ↑ (%) | 98.1 | 95.7 | 76.8 | 41.0 | 20.6 | 10.5 | 76.3 | 89.7 | |
| ↓ (s) | 493 | 745 | 1415 | 2101 | 3636 | 7965 | 1802 | 1177 | |
| MATH-500 | Acc. ↑ | 26.2 | 34.4 | 41.8 | 46.4 | 46.0 | 44.8 | 45.2 | 44.8 |
| 62.4 | 121.0 | 222.7 | 329.0 | 402.4 | 590.0 | 260.0 | 221.8 | ||
| 64 | 128 | 256 | 512 | 1024 | 2048 | 333.7 | 241.8 | ||
| ↑ (%) | 97.5 | 94.6 | 87.0 | 64.3 | 39.3 | 28.8 | 77.9 | 91.8 | |
| ↓ (s) | 176 | 330 | 635 | 1160 | 2025 | 4400 | 1036 | 645 | |
| MBPP | Acc. ↑ | 6.6 | 40.0 | 49.4 | 48.4 | 50.6 | 51.4 | 49.6 | 52.2 |
| 58.5 | 114.3 | 168.2 | 191.3 | 199.0 | 247.6 | 159.7 | 170.7 | ||
| 64 | 128 | 256 | 512 | 1024 | 2048 | 288.8 | 226.0 | ||
| ↑ (%) | 91.4 | 89.3 | 65.7 | 37.4 | 19.4 | 12.1 | 55.3 | 75.5 | |
| ↓ (s) | 254 | 449 | 717 | 1032 | 1704 | 3617 | 1381.5 | 916.5 | |
| HumanEval | Acc. ↑ | 9.2 | 32.3 | 56.1 | 53.7 | 51.8 | 55.5 | 54.9 | 55.5 |
| 56.9 | 121.1 | 185.9 | 241.8 | 254.1 | 268.4 | 185.5 | 180.7 | ||
| 64 | 128 | 256 | 512 | 1024 | 2048 | 310.7 | 235.9 | ||
| ↑ (%) | 88.9 | 94.6 | 72.6 | 47.2 | 24.8 | 13.1 | 59.7 | 76.6 | |
| ↓ (s) | 92 | 162 | 262 | 431 | 663 | 1340 | 419 | 268 |
MMaDA:
| Benchmark | Metric | 64 | 128 | 256 | 512 | 1024 | 2048 | DAEDAL | -EOS [,] |
|---|---|---|---|---|---|---|---|---|---|
| GSM8K | Acc. ↑ | 43.0 | 62.5 | 67.4 | 68.0 | 70.2 | 71.2 | 70.8 | 70.9 |
| 63.5 | 115.1 | 178.1 | 225.2 | 300.4 | 433.5 | 130.9 | 151.2 | ||
| 64 | 128 | 256 | 512 | 1024 | 2048 | 171.7 | 179.7 | ||
| ↑ (%) | 99.3 | 90.0 | 69.6 | 44.0 | 29.3 | 21.2 | 76.2 | 84.1 | |
| ↓ (s) | 298 | 627 | 1347 | 2848 | 5862 | 15056 | 1277 | 1061 | |
| MATH-500 | Acc. ↑ | 20.9 | 25.1 | 34.6 | 34.8 | 35.0 | 35.4 | 34.6 | 35.0 |
| 58.7 | 106.4 | 187.8 | 296.1 | 440.5 | 647.4 | 130.8 | 123.1 | ||
| 64 | 128 | 256 | 512 | 1024 | 2048 | 181.9 | 145.3 | ||
| ↑ (%) | 91.7 | 83.1 | 73.4 | 57.8 | 43.0 | 31.6 | 71.9 | 84.7 | |
| ↓ (s) | 205 | 374 | 780 | 1795 | 4375 | 10361 | 1074 | 643 |
Response to Reviewer qFnx: Part 3
For the block-diffusion models, LLaDA-2.0-mini and SDAR-8B-Chat-b32, we apply -EOS at the block level. Both models have an inherent block size of 32. With -EOS, the canvas size of each block is no longer fixed at 32; instead, it is adaptively adjusted according to the implicit EOS density.
LLaDA-2.0-mini:
| Benchmark | Metric | Baseline | -EOS [,] |
|---|---|---|---|
| GSM8K | Acc. ↑ | 92.3 | 89.2 |
| 330.6 | 306.4 | ||
| ↓ (s) | 6503 | 5704 | |
| MATH-500 | Acc. ↑ | 69.8 | 69.8 |
| 946.0 | 934.8 | ||
| ↓ (s) | 6894 | 5250 | |
| MBPP | Acc. ↑ | 69.2 | 68.8 |
| 743.0 | 708.9 | ||
| ↓ (s) | 7007 | 4988 | |
| HumanEval | Acc. ↑ | 80.5 | 80.7 |
| 667.7 | 713.5 | ||
| ↓ (s) | 1791 | 1388 |
SDAR-8B-Chat-b32:
| Benchmark | Metric | Baseline | -EOS [,] |
|---|---|---|---|
| GSM8K | Acc. ↑ | 87.7 | 88.9 |
| 228.7 | 229.3 | ||
| ↓ (s) | 873 | 662 | |
| MATH-500 | Acc. ↑ | 59.8 | 59.8 |
| 500.6 | 496.2 | ||
| ↓ (s) | 738 | 658 | |
| MBPP | Acc. ↑ | 44.8 | 43.0 |
| 148.8 | 134.3 | ||
| ↓ (s) | 302 | 225 | |
| HumanEval | Acc. ↑ | 68.3 | 68.0 |
| 219.4 | 187.9 | ||
| ↓ (s) | 117 | 99 |
Overall, these results demonstrate that -EOS generalizes to a broad range of masked dLLMs, including LLaDA, Dream, and MMaDA. For the block-diffusion models (LLaDA-2.0 and SDAR), their semi-autoregressive generation process means that they are not subject to the same fixed-canvas limitation. Consequently, -EOS does not consistently improve task accuracy for these models. Nevertheless, its ability to contract the canvas reduces the number of generated tokens in many settings and yields consistent runtime reductions across the reported block-diffusion experiments.
Response to Reviewer qFnx: Part 4
About Q4: Contraction safeguard
We thank the reviewer for this constructive suggestion. Based on the failure mode observed in Table 10, we introduce two safeguards to prevent the inadvertent truncation of semantically meaningful content. The first, safeguard-non-EOS, halts contraction upon encountering the first non-EOS token when scanning from the end of the canvas; it may therefore remove only trailing [EOS] or [MASK] tokens. The second, safeguard-committed, permits the removal of trailing [MASK] tokens only, ensuring that no committed token, including an EOS token, can be truncated. The latter will be the conservative default in the revised method, whereas safeguard-non-EOS is reported as a more aggressive optional efficiency-oriented variant.
We evaluate both safeguards under the same settings as in Table 9. The results are shown below.
| Contraction | GSM8k | MATH-500 | MBPP | Humaneval | ||||
|---|---|---|---|---|---|---|---|---|
| Acc. | remove generated token event ratio | Acc. | remove generated token event ratio | Acc. | remove generated token event ratio | Acc. | remove generated token event ratio | |
| Enabled | 81.8 | 50.23% | 37.0 | 23.60% | 39.6 | 10.60% | 43.3 | 6.71% |
| Disabled | 83.1 | 0% | 38.0 | 0% | 39.8 | 0% | 45.1 | 0% |
| safeguard-non-EOS | 83.2 | 0% | 38.0 | 0% | 39.7 | 0% | 45.1 | 0% |
| safeguard-committed | 83.2 | 0% | 38.2 | 0% | 39.8 | 0% | 45.1 | 0% |
As shown above, unconstrained contraction (“Enabled”) yields the lowest accuracy on all four benchmarks. Both safeguards reduce the generated-token-removal event ratio to zero, matching the result of disabling contraction. In particular, safeguard-committed matches or improves upon the accuracy of disabled contraction across all four benchmarks. safeguard-non-EOS is similarly comparable, with only a 0.1-point decrease on MBPP and a 0.2-point decrease on MATH-500. These results provide evidence that the proposed safeguards effectively eliminate undesirable token-removal events while preserving task performance.
Response to Reviewer qFnx: Part 1
About W1 & Q2: Quality-efficiency and Pareto claim
We thank the reviewer for this helpful suggestion. We will revise the Pareto-related claims in Section 4.3 and in the conclusion. Specifically, we will replace broad statements such as “Pareto dominates” and “superior Pareto frontier” with the more precise characterization that our method provides a favorable quality–efficiency trade-off, with Pareto dominance restricted to the latency-dominated portion of the frontier. We will also explicitly report the joint accuracy–latency Pareto frontier across all methods.
About W2 & Q1 - Reproducibility and scope gaps (decoding configuration)
We thank the reviewer for this constructive comment. In the current manuscript, Section 4.1 (“Implementation Details”) summarizes the decoding configuration—including the confidence threshold τhigh, block length, and denoising/remasking schedule—with the statement: “We follow the settings of DAEDAL (Li et al., 2025a) to ensure fairness and reproducibility.” To make all choices explicit and reproducible, we will replace this statement with the configuration table below.
steps=gen_length[0.2, 0.8][0.4, 0.8]steps=gen_length[0.2, 0.8][0.4, 0.8]steps=gen_length[0.4, 0.8]As the request of the reviewer, to quantify run-to-run variation, we additionally report results over three runs. The following values are reported as mean ± standard deviation.