acceptodds
Under review as a conference paper at ICLR 2027

TAPER: Token Admission with Priced Early Release for Masked Diffusion Decoding

Abstract

Without a key–value cache, every call of a blockwise masked diffusion decoder predicts all masked positions, yet only the active block may be committed. We study (Token Admission with Priced Early Release), a training-free rule that also commits low-risk positions in a bounded window beyond the block by reusing the active block's price. Released positions never count toward the block's completion quota, and the rule adds no model call, history buffer or calibration table. An exact per-output ledger, , shows why early commits are not saved calls: they mostly replace later within-block batching, so the aggregate call reduction per early commit is only 0.09–0.12. On MATH500, GSM8K, HumanEval and MBPP, windows chosen on these benchmarks cut calls by 6.1% (LLaDA-8B) and 7.9% (Dream-7B) at the same price, with paired accuracy changes of [, ] and [, ] points; the reductions replicate across hardware, prices, lengths and 1,142 development problems. Controls locate the saving: with end-of-sequence tail completion applied to both policies, release still removes about 6% of calls, and at equal admission a confidence-and-history gate with AHD's released thresholds saves little because of its history condition, while confidence alone recovers most of the saving, which limits claims for the price gate itself. Release is a second knob rather than a better operating curve: a fixed price does not fix an accuracy target, we establish neither accuracy non-inferiority nor a reproducible advantage over retuning the admission price, and at length 128 accuracy point estimates fall by up to about one point.

Primary area:
generative models
Keywords:
masked diffusion language models, unmasking policy, train-free inference, parallel decoding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cnUkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TAPER, a training-free rule for speeding up blockwise decoding in masked diffusion language models (LLaDA and Dream). In standard blockwise decoding, every model call predicts all masked positions, but only positions in the current block can be committed. TAPER also commits positions near the start of the next block when the model is already confident about them, using the same threshold as the current block. It needs no extra model calls or stored history. The authors also show with a simple counting identity that the number of tokens committed early is not the number of calls saved. Most early commits only replace tokens that would otherwise have been committed together later in their own block. On four math and code benchmarks, TAPER reduces the number of model calls by a few percent with little change in accuracy, although the authors acknowledge that retuning the threshold of the existing commit rule gives almost the same trade-off.

Critical strengths

The paper is careful about what its method does and does not do. The counting identity is simple, but the point behind it is useful. Methods that commit tokens early should report how many calls are actually saved, not how many tokens were committed early. The paper also includes many controls that try to locate where the saving comes from, such as handling end-of-sequence tokens separately, releasing positions by distance instead of confidence, and splitting the AHD gate into its confidence and history parts. I also appreciate that negative results are reported openly, including that the method does not beat retuning the threshold and that the accuracy non-inferiority is not established.

Critical weaknesses
  1. The contribution is small. The call reduction is a few percent, and the authors themselves say that retuning the threshold of the existing commit rule gives a similar trade-off. Reusing the block's threshold also does not seem essential, since a plain confidence threshold recovers most of the savings. The counting identity is bookkeeping and is not enough on its own to count as a main contribution.
  2. All results assume decoding without a key-value cache, where every call costs the same. In practice, caching and partial recomputation are widely used, and in that setting, fewer calls do not necessarily mean less compute or less time. The paper mentions this in the limitations, but does not measure it.
  3. Several training-free methods that also reduce the number of decoding steps or relax block boundaries are neither cited nor compared, for example, dilated scheduling (Luxembourg et al., 2025), SlowFast (Wei et al., 2025), and CtrlDiff (Huang & Tang, 2025).
  4. The window size and threshold were chosen on the same benchmarks used for testing, and the advantage over retuning the threshold at length 512 did not hold up in the pre-planned evaluation on development problems. The paper is also hard to read. The main text is packed with numbers and qualifications, and it took me a while to work out what the central claim is.
Questions for authors
  1. When TAPER is combined with key-value caching or methods that recompute only some positions, how much of the call reduction remains as a reduction in compute or wall-clock time?
  2. If retuning the threshold gives almost the same trade-off, in what situation would a practitioner choose TAPER over simply adjusting the threshold?
  3. How does TAPER compare with dilated scheduling, SlowFast or CtrlDiff in accuracy at a matched number of calls?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer cnUk (part 1 of 2)

Thank you for the clear questions.

TL;DR. (Q1) Yes: under Fast-dLLM's dual cache release lowers wall-clock time by 4–14%, and a cost ledger shows why (a removed call saves a cached call; a block filled before it opens skips its full-forward refresh). (Q2) Under a cache, release saves time on a threshold as it stands; against our retuned margin price it is more accurate at equal time on Dream (interval excludes zero) and only in point estimate on LLaDA, and on Fast-dLLM's threshold the gap to retuning is within noise. Without a cache it adds operating points next to retuning. (Q3) DUS and SlowFast's official settings are 6.7–30 points below our frontier at matched calls (Table 3). (W4) Development-only selection picks the same LLaDA window, and the L=512L=512 gain over retuning is no longer claimed.

Q2 / W1. Why choose release over retuning the threshold? We agree with two of your points: our price is not essential, and the counting identity alone is bookkeeping. What remains is a release step that adds no call and no threshold to any threshold rule, the measurement that one early commit is worth about 0.1 calls, and a wall-clock saving of 4–14% under a key–value cache with a cost ledger that explains it.

  • Under a key–value cache, release is ahead of our retuned margin price on Dream, and in point estimate on LLaDA. With the margin price retuned to σ=6\sigma=6 and 8 under Fast-dLLM's dual cache (L=256L=256, full test sets; Tables A24 and A25), σ=6\sigma=6 with release is, at equal time, +2.38 [+0.36, +4.39] points above σ=8\sigma=8 on Dream and +0.72 [−1.35, +2.83] on LLaDA. Times come from the cost ledger, which a one-session timing check matches to within 1.7 points (Appendix F.3). On Fast-dLLM's own threshold the gap is within noise: 0.90 with release against 0.85 is +1.30 [−0.65, +3.21] (Dream) and +0.34 [−1.45, +2.25] (LLaDA) points at 5.6% and 4.1% more time (Table A25, last rows).
  • At equal query width, the early commits still remove time. Most of the accuracy gain above comes from the wider query that release needs (+1.74 points on Dream and +1.19 on LLaDA at σ=6\sigma=6). Against a control that only widens the cached query, the early commits remove 4.4–7.5% of time at L=256L=256 (two rules, both models) and 6.5% and 13.8% at L=512L=512, and all six paired accuracy intervals contain zero (Table A23; last column of the table in part 2). At σ=6\sigma=6 they change accuracy by −0.16 [−1.67, +1.27] (Dream) and −1.21 [−2.92, +0.57] (LLaDA) points (Table A25).
  • Without a cache, it adds operating points next to retuning. Of the 11 non-dominated operating points of the five admission settings, 6 use release (all four on Dream, two of seven on LLaDA). On LLaDA, admission at σ=6\sigma=6 dominates our σ=4\sigma=4 with release, and release still removes 7.4% of σ=6\sigma=6's calls (Tables A19, A8), so both knobs should be swept.
  • It needs no new threshold. Release keeps the rule's criterion and quota and adds only window positions that pass the same test. On Fast-dLLM (τ=0.9\tau=0.9) it removes 5.1% (LLaDA) and 6.2% (Dream) of calls, and under a dual cache 7.4–8.2% at L=256L=256 and 8.6–10.5% at L=512L=512 (Tables A18 and 6). Release works with any threshold rule (Table 2); it does change outputs (Appendix F.1), so the next point applies.
  • Accuracy still has to be checked. On the five settings every paired interval contains zero (Table A18). LLaDA's point estimates are all slightly negative; pooled over the five settings, which share their items, the change is −0.38 [−0.90, +0.14] points on LLaDA and +0.26 [−0.29, +0.84] on Dream. An interval that contains zero does not show that there is no loss; what these exclude, at the 95% level, is a loss above 0.90 points on LLaDA and above 0.29 on Dream. LLaDA releases mostly content, and its flips balance (59 to wrong, 45 to correct, p=0.20p=0.20; Appendix F.1). Elsewhere intervals exclude zero for a gain under the cache on Dream (+2.08, Table 6) and for losses with a full-block window at L=128L=128 and on LLaDA's factor rule at L=512L=512 (Figure A1, Table A18).
  • What the ledger adds. We agree that the identity alone is bookkeeping. Its use is measurement: one early commit removes only 0.07–0.12 calls on every admission rule, and less under AHD-style history gates (Tables 4, A18, A30), so counting early commits overstates the saving about tenfold. Extended to the cache, the same accounting predicts time (Table A26). Release also stacks on other accelerations: with it, Fast-dLLM factor is the fastest decoder we timed (Table 5).
AuthorsOpenReview ↗

Reply to Reviewer cnUk (part 2 of 2)

Q1 / W2. Does the saving survive a KV cache? Yes (Section 4.6; Tables 6, A23 and A26). Under Fast-dLLM's dual cache (RTX4090, full test sets, paired), release processes 13–26% more query tokens per output at L=256L=256, yet it removes 7.4–10.0% of calls and 3.8–8.2% of wall-clock time there, and 8.6–11.8% and 8.2–13.6% at L=512L=512:

Model, admission ruleNo release: acc / calls / sRelease: acc / calls / sΔ\Delta acc [95% CI]Δ\Delta calls / Δ query / Δ timeEqual query width: Δ\Delta acc / Δ\Delta calls / Δ time
LLaDA, margin price46.91 / 85.9 / 3.1348.59 / 77.9 / 2.87+1.68 [−0.28, +3.70]−9.3% / +13.0% / −8.2%+0.06 / −5.9% / −4.8%
LLaDA, Fast-dLLM (τ=0.90\tau=0.90)48.63 / 89.4 / 3.2248.86 / 82.0 / 2.99+0.23 [−1.61, +2.06]−8.2% / +14.2% / −7.2%−0.09 / −5.3% / −4.4%
Dream, margin price59.78 / 83.2 / 2.3661.86 / 74.9 / 2.20+2.08 [+0.13, +4.01]−10.0% / +22.0% / −6.6%+0.28 / −7.0% / −7.5%
Dream, Fast-dLLM (τ=0.90\tau=0.90)61.90 / 86.4 / 2.4862.51 / 80.1 / 2.38+0.61 [−1.30, +2.50]−7.4% / +26.0% / −3.8%+0.66 / −5.5% / −5.8%
LLaDA, margin price, L=512L=51252.56 / 140.3 / 6.3153.17 / 126.7 / 5.77+0.61 [−1.25, +2.58]−9.7% / +8.7% / −8.5%−0.13 / −6.5% / −6.5%
LLaDA, Fast-dLLM (τ=0.90\tau=0.90), L=512L=51253.15 / 145.5 / 6.6953.32 / 132.9 / 6.15+0.17 [−1.87, +2.27]−8.6% / +9.5% / −8.2%not run
Dream, margin price, L=512L=51262.00 / 101.6 / 3.9562.82 / 89.7 / 3.41+0.82 [−1.09, +2.77]−11.8% / −7.9% / −13.6%−0.10 / −10.3% / −13.8%
Dream, Fast-dLLM (τ=0.90\tau=0.90), L=512L=51263.05 / 104.7 / 4.0464.95 / 93.7 / 3.60+1.89 [0.00, +3.79]−10.5% / −6.3% / −10.9%not run

Why the wider query does not eat the saving (cost ledger, Table A26): fitted to the per-output rows (R2≥0.99R^2 \geq 0.99), a full forward costs 1.4–2.2 cached calls, and widening the query by h adds only 0.5–6% to a cached call. Each removed call therefore saves almost a whole cached call, and a block that release fills before it opens skips its full forward. On Dream at L=512L=512, of the 13.2% saving the ledger gives 9.2 points to skipped full forwards (10.9 instead of 16 per output) and 5.5 to fewer cached calls; the wider query costs 1.4. LLaDA's window is shorter than a block, so it saves through fewer calls alone. The last column is a control that widens the query without releasing: it attributes the accuracy rise to the refreshed window (+1.62 and +1.80 points on the margin price), and at equal query width release still removes 4.4–7.5% of time at L=256L=256 and 6.5–13.8% at L=512L=512, with every accuracy interval containing zero (Table A23). We tested one cache design, with batch size one.

Q3 / W3. DUS, SlowFast and CtrlDiff at matched calls. All three are now cited (Section 5). DUS and SlowFast are training-free, so we ran them in our pipeline (Table 3; the other block sizes in Table A21):

MethodLLaDA acc / callsLLaDA gapDream acc / callsDream gap
TAPER47.72 / 71.4−0.5561.90 / 74.30.00
DUS, b = 843.06 / 96–51.17 / 96–
DUS, b = 1641.05 / 64−6.7046.71 / 64−14.85
DUS, b = 3235.96 / 40–34.85 / 40–
SlowFast, official41.87 / 87.1−7.4032.12 / 80.9−29.97
SlowFast, 32-token blocks10.87 / 69.4−37.3049.15 / 99.6–
SlowFast, 32-token blocks, full canvas49.14 / 99.8–61.48 / 108.9–

Gap: accuracy minus our frontier of the five admission settings at the method's calls; a dash means the calls lie outside the frontier's range. SlowFast ends most outputs early with 32-token blocks on LLaDA and with its blockless sampler on Dream. The official SlowFast code, run unmodified on 64 GSM8K items, matches our port to within one item. With 32-token blocks the early endings come from the official prefix-only fast-phase calls: computed on the full canvas, SlowFast reaches 49.14% (LLaDA) and 61.48% (Dream), +1.41 [−0.42, +3.19] and −0.42 [−2.24, +1.32] points against TAPER, at 40% and 47% more calls. CtrlDiff trains its own small model from scratch with an RL block-size policy and reports perplexity; we found no public code, so we cite it but cannot run it on LLaDA or Dream.

W4. Settings chosen on the test benchmarks; readability. On the 1,142 development problems alone, the selection rule picks h=16h=16 for LLaDA (the test-set choice) and h=8h=8 for Dream; the three windows differ by less than one point on either set (Table A27), and the call reduction holds on these problems (Table A7). σ=4\sigma=4 was also fixed during earlier development on these benchmarks, and Section 2 now says so; release saves 5.6–7.4% of calls at every σ\sigma from 3 to 6 (Table A8). The L=512L=512 gain over retuning, which the pre-specified development evaluation did not reproduce (+0.24 and +0.01 points), is no longer claimed. Section 4 now follows four questions, and the numbers are in tables.

Reviewer_Y69POpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes TAPER, a training-free decoding rule for blockwise masked diffusion LMs. Besides the active block, it also commits low-risk tokens in a short window ahead of it, using the same threshold as the block’s own admission rule and without extra forward passes. The contributions are:

  1. The release rule itself, with guarantees that every block still finishes within its budget.
  2. An exact step count, NFE=L−E−SNFE = L - E - S (L: output length, E: early commits, S: extra tokens committed in parallel within a block), showing that early commits mostly replace tokens that would have been committed in parallel later, so each saves only about 0.1 forward passes.
  3. Controls that locate where the saving comes from: end-of-sequence handling, a blind release baseline, and a history-based gate.

On LLaDA-8B and Dream-7B across four math and code benchmarks, TAPER reduces forward passes by about 6–8% with no clear change in accuracy.

Critical strengths

A clear way to measure what early decoding actually saves. The paper breaks down where each committed token comes from, and shows that tokens committed early mostly take the place of tokens that would have been committed in parallel later anyway. In practice, about ten early commits save only one forward pass. Reporting early commits alongside the actual saved passes is an easy practice for others to adopt.

Simple and training-free, with clear guarantees and solid controls. The release rule doesnt need extra forward passes or retraining, and every block still finishes within its budget. Controls such as end-of-sequence handling, blind release and a history-based gate help show where the saving comes from, using paired comparisons on the same prompts.

Critical weaknesses
  1. Writing and structure

The paper reads as heavily machine-written: many sentences are built around hedges and negations (“this does not mean that…”), and the text does not flow. The intro and method sections do not clearly present the problem, the rule and the contributions. The results section reports scores without telling a story. In addition, appendix tables should be organized properly and not referred as a content available in the main body ("Referring to Appendix ...").

  1. No comparison with admission-and-release schedulers

TAPER is positioned among cross-block early-commit methods (D2F, AHD, AdaBlock-dLLM), but it is compared only against block-local schedulers. AHD appears only as a gate ablation, and D2F and AdaBlock-dLLM are not evaluated. The baselines are also not shown everywhere TAPER is: for example, the bottom panels of Fig. 2 (L = 512) show only TAPER and admission, while the top panels include the external samplers.

Questions for authors

Questions:

  1. Can you compare TAPER with admission-and-release methods (AHD as a full method, AdaBlock-dLLM, and D2F if checkpoints are available), using the same metrics and marking which methods require training?

  2. Can you report tokens per second (TPS) for all methods, including Fast-dLLM and ADAS, and bring the timing results (Table 20) into the main text? Together with the forward-pass counts, this would show the actual speed of each method.

Suggestions:

  1. It would be interesting to see the share of end-of-sequence releases (“Terminal” in Table 3) for all window sizes, not only h = 32. Released tokens that stay fixed could be a good indication of real savings.
  2. I could not find the reasoning behind σ = 4. A short justification or a pointer to an ablation might help. The timing results (Table 20) may be more useful to readers in the main text than the L ablation, which could fit in the appendix.
  3. Fig. 2 seems more informative to the reader than Table 1. Presenting it first may make the main trade-off easier to see.
  4. When referring to the appendix, a short note on what each table shows might be easier to follow than a list of table numbers.

A clearer presentation of the method, a comparison with admission-and-release methods, and moving key results such as TPS and the Terminal analysis into the main text would lead me to raise my score.

5 comments
AuthorsOpenReview ↗

Reply to Reviewer Y69P (part 1 of 3)

Thank you for the specific suggestions. You named three changes that would raise your score:

Your conditionWhat we didWhere
A clearer presentation of the method [W1, S2–S4]Rewrote the abstract, introduction and Sections 2–6; Section 4 follows four questions; Figure 2 on page 3main text
Comparison with admission-and-release methods [W2, Q1]AHD (full method), AdaBlock-dLLM with and without release, D2F, DUS and SlowFast in our pipeline; every decoder at L=512L=512Table 3; Figure 2, bottom row
TPS and the Terminal analysis in the main text [Q2, S1]Tokens per second for all 14 decoders; terminal share for every windowTables 5 and 4

Details follow under the same three headings, with each of your items in brackets.

1. A clearer presentation (W1, S2–S4).

  • [W1] Writing and structure. We rewrote the abstract, introduction and Sections 2–6. The introduction now states the problem with one measurement (under admission alone, 61% (LLaDA) and 68% (Dream) of calls commit a single token, and on 24% of calls a position in the next 32 already has a risk below the active block's price) and lists three contributions. Section 4 opens with the four questions it answers, and each results subsection opens with its finding, e.g. "Most early commits replace later batching". Intervals are in the tables, and the caveats are in one Limitations paragraph.
  • [S3] Figure 2 first. Figure 2 is cited in the introduction and placed on page 3, before Table 1.
  • [S2] σ=4\sigma=4, and timing before the length ablation. The length figure moved to Appendix C.3 (Figure A1), and the timing results (old Table 20) are Table 5 in Section 4.6. We fixed σ=4\sigma=4 during earlier development on the same benchmarks, and Section 2 now says so. It acts as a compute setting: a constant price σ/H\sigma/H at σ=4\sigma=4, 6 and 8 lies on the same accuracy–calls curve as the rising price (gaps −0.19 to +0.25 points, Table A29), and release saves 5.6–7.4% of LLaDA's calls at every price from σ=3\sigma=3 to 6 (Table A8).
  • [W1, S4] Appendix references. Appendix tables are numbered A1–A31, and every reference to them, in the text and in captions, carries a short note on what the table shows, e.g. "blind control, Table A4" or "Per-rule results: Table A18; sampled decoding: Appendix F.1; cache control: Table A23"; no reference is a bare list of numbers. A contents page (page 16) lists every section of the main text and the appendix.

2. Comparison with admission-and-release methods, now in the main text (W2, Q1).

  • [W2, Q1] Same metrics, training marked. Same GPU type, full test sets, L=256L=256, our prompts and scorer (Table 3; further DUS and SlowFast settings in Table A21):
MethodTrainingLLaDA acc / calls / tokens/sDream acc / calls / tokens/s
TAPERnone47.72 / 71.4 / 71.261.90 / 74.3 / 68.0
Fast-dLLM (τ=0.90\tau=0.90) + releasenone48.41 / 74.5 / 67.161.99 / 79.6 / 63.6
AHDnone48.28 / 78.5 / 45.562.29 / 78.4 / 42.1 (our port)
AdaBlock-dLLMnone50.07 / 84.2 / 57.761.27 / 85.8 / 56.6
AdaBlock-dLLM + our releasenone49.32 / 79.0 / 61.561.83 / 82.1 / 59.2
D2F (L=512L=512, own cache)LoRA44.19 / 96.3 / – (3.26 s per output)not run (its Dream LoRA is built on Dream-Base)
  • What ran where, and what the table shows. AHD and AdaBlock-dLLM are training-free, so both run on LLaDA and on Dream (AHD's official code supports LLaDA only; its Dream row is our port). D2F needs a trained LoRA, and its released Dream LoRA is built on Dream's base model, not on the Instruct checkpoint we evaluate, so D2F runs on LLaDA only. Among the training-free rows above, TAPER uses the fewest calls and decodes the most tokens per second on both models. It is not the most accurate: AdaBlock-dLLM is on LLaDA and AHD on Dream. Release does not compete with these schedulers; added to AdaBlock-dLLM it removes 4–6% of its calls.
  • [Q1] AdaBlock-dLLM. It is the most accurate LLaDA setting: +0.98 [−0.33, +2.30] points over the same in-block rule (Fast-dLLM (τ=0.90\tau=0.90)) at 7.2% more calls; on Dream it is 0.53 points below that rule. It moves block boundaries rather than eligibility, so release composes with it: −6.1% [−6.7, −5.5] calls on LLaDA and −4.4% [−4.8, −4.0] on Dream, at −0.75 [−2.03, +0.51] and +0.57 [−0.38, +1.56] points.
  • [Q1] AHD as a full method. On Dream, AHD is +0.30 [−1.31, +1.96] points above Fast-dLLM (τ=0.90\tau=0.90) + release at 1.5% fewer calls, but its float64 history makes each call about 1.4 times as long, so on Dream it decodes 42.1 tokens/s against 63.6 for that rule and 68.0 for TAPER, and it needs 18.1 GiB of peak memory against 14.7 GiB. On LLaDA, Fast-dLLM (τ=0.90\tau=0.90) + release dominates AHD (48.41 / 74.5 against 48.28 / 78.5).
AuthorsOpenReview ↗

Reply to Reviewer Y69P (part 2 of 3)

2. Comparison with admission-and-release methods (continued).

  • [Q1] D2F, which requires training. D2F trains a LoRA and decodes pipelined blocks with its own block-causal model and cache. We ran its released LLaDA-8B-Instruct LoRA with D2F's official per-dataset settings at its official length of 512 new tokens (at 256, 38–86% of its outputs end with no end-of-sequence token). Against TAPER at L=512L=512 on the same card, both with a cache (TAPER with Fast-dLLM's dual cache), it changes accuracy by −8.98 [−11.25, −6.78] points and takes 3.26 against 5.77 seconds per output, so it is 1.8 times as fast. Without a cache TAPER takes 10.10 seconds. Its largest gap is on HumanEval; without HumanEval it is still about 6 points below TAPER. Its calls run on cached states and are not full forwards, so time is its comparable cost (Table A22).
  • [W2] Baselines at L=512L=512. Figure 2's bottom row now shows every decoder at L=512L=512 on one RTX4090 type: the margin price, TAPER, Fast-dLLM's two rules with and without release, ADAS, KLASS and LESS. Its former content, the price sweep of the main runs, is in Table A5. Values (Tables A18 and A20; acc / calls, release minus no release, paired):
Rule (L=512L=512)LLaDA: no release → + releaseLLaDA Δ\Delta acc [95% CI] / Δ\Delta callsDream: no release → + releaseDream Δ\Delta acc [95% CI] / Δ\Delta calls
Margin price σ=4\sigma=4 (+ release = TAPER)54.20 / 130.7 → 54.09 / 121.2−0.11 [−1.27, +1.05] / −7.3%62.47 / 98.8 → 62.70 / 87.2+0.24 [−0.69, +1.17] / −11.7%
Fast-dLLM (τ=0.90\tau=0.90)54.01 / 136.1 → 53.05 / 128.2−0.96 [−2.01, +0.12] / −5.8%62.90 / 103.0 → 62.88 / 93.0−0.01 [−1.04, +0.98] / −9.7%
Fast-dLLM factor53.51 / 107.4 → 52.16 / 99.9−1.35 [−2.72, −0.001] / −7.0%62.87 / 82.2 → 62.43 / 73.1−0.45 [−1.56, +0.64] / −11.1%
Margin price σ=8\sigma=8 (no release)52.06 / 104.6—62.45 / 82.2—
ADAS + Fast-dLLM factor53.55 / 109.7—61.74 / 83.7—
KLASS53.20 / 209.2—63.84 / 179.8—
LESS52.44 / 166.8—64.12 / 141.8—

Release lowers calls on every rule, and five of the six accuracy intervals contain zero; on LLaDA's Fast-dLLM factor it costs 1.35 points, which we report as a cost of release on that rule. Fast-dLLM factor is strong at this length: on Dream it alone is slightly more accurate than TAPER at fewer calls. ADAS, KLASS and LESS keep the paper's settings and reproduce their L=256L=256 timing rows. KLASS and LESS use 1.6–2.1 times TAPER's calls; they are less accurate than TAPER on LLaDA and more accurate on Dream (by 1.1 and 1.4 points, mostly on HumanEval), and every paired interval contains zero. Four of the ten non-dominated settings use release.

AuthorsOpenReview ↗

Reply to Reviewer Y69P (part 3 of 3)

3. TPS and the Terminal analysis in the main text (Q2, S1, S2).

  • [Q2, S2] Tokens per second for all methods, in the main text. Table 5 (old Table 20) now times all fourteen decoders, including Fast-dLLM and ADAS, in one session, without a cache, on the same items and card (tokens per second):
Rule or methodLLaDA: alone / + releaseDream: alone / + release
Margin price (+ release = TAPER)66.3 / 71.262.3 / 68.0
Fast-dLLM (τ=0.90\tau=0.90)63.4 / 67.159.7 / 63.6
Fast-dLLM factor80.7 / 86.776.1 / 81.2
AdaBlock-dLLM57.7 / 61.556.6 / 59.2
ADAS + Fast-dLLM factor72.867.3
KLASS40.632.4
LESS50.947.3
AHD45.542.1
DUS (b=16b=16)78.880.4
SlowFast56.273.2

Release lowers time on every rule it joins by 4.5–8.4%, about as much as it lowers calls, with peak memory within 0.1 GiB; with release, Fast-dLLM factor is the fastest decoder we timed. Each method runs at its own accuracy (Tables 1 and 3): Fast-dLLM and DUS are faster than TAPER and less accurate, and AHD takes 46–48% longer than our admission at similar calls. Table 6 adds Fast-dLLM's dual key–value cache, where release cuts 3.8–8.2% of time at L=256L=256 and 8.2–13.6% at L=512L=512.

  • [S1] Terminal share for every window. Table 4 now gives the terminal share and the content early commits for h=8h=8, 16 and 32:
h=8h=8h=16h=16h=32h=32
LLaDA: terminal share / content early commits per output8.8% / 29.211.4% / 44.914.8% / 56.3
Dream: terminal share / content early commits per output44.5% / 18.054.7% / 24.658.3% / 28.8

Reading the table: the share of early commits that are end-of-sequence padding grows with the window, from 9% to 15% on LLaDA and from 45% to 58% on Dream. Dream's saving therefore leans on padding far more than LLaDA's, and LLaDA releases about twice as much answer content per output (56.3 against 28.8 tokens at h=32h=32). The saving is still more than an early tail: releasing only end-of-sequence proposals removes 0.1% (LLaDA) and 2.7% (Dream) of calls, against 6.1% and 7.9% for TAPER (Table A14).

  • [S1] Released tokens stay fixed. Released tokens are never remasked, so every one of them stays fixed. What matters is whether they match what admission alone would have committed: at h=32h=32, 85.8% (LLaDA) and 84.8% (Dream) of released content tokens and 95.1% and 96.2% of released end-of-sequence tokens equal the admission-only token at the same position, and about half of the outputs (49.3% and 49.7%) are identical (Section 4.5; "What release keeps", Appendix F.1). With online tail completion on both policies, release still removes 6.1% and 6.2% of calls, and the remaining early commits, mostly content, remove 0.11 and 0.15 calls each, more than the 0.09–0.12 of all early commits (Table A14, Appendix D.2).
Reviewer_F9E1OpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Existing blockwise masked diffusion decoders waste computation as every call scores all masked positions but only commits one block’s worth of tokens, and the the admission price , i.e., confidence threshold for committing tokens within the block are silently discarded and requires recompution. To improve the decoding efficiency, the paper proposes a new strategy: when a token just outside the block looks sufficiently confident (low-risk), it gets committed early via reusing the confidence score already computed for the active block. This is expected to be effective as these early releases don’t count toward the current block’s completion quota, and the method adds no extra model calls, no history buffer, and no calibration table.

Empirical results show that across multiple datasets, it has reduced the model call without losing the accuracy.

Critical strengths
  1. The limitation of existing methods are clear, and the problem motivation is strong.

  2. The empirical results are somewhat strong.

Critical weaknesses
  1. From the results, it seems that the confidence alone recovers most of the gain, which is a common strategy. See Table10, ablation results: The confidence-only ablation nearly matches the proposed method.

  2. There are lots of variables, formulations, but they are not consistent in section 2 (preliminary) and section 3 (methods). These notation feels like it was developed in pieces rather than designed as a unified framework, with section 3 grafting new variables onto section 2 rather than inheriting cleanly from it.

Questions for authors
  1. Can you elaborate the effects of "price gate", showing how it meaningfully outperforms a tuned fixed threshold?

  2. In figure3. the performance on shorter generation is worse than baselines. Can you explain that? and any solutions?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer F9E1 (part 1 of 2)

Thank you for the review.

TL;DR. (W1) Your observation is right, and it changed the paper: the window supplies the saving, so release is now presented as a step for any threshold rule (Table 2). (W2) One notation runs from Section 2 through Algorithm 1. (Q1) In calls, slightly; in accuracy, not resolvably: the price gate uses 0.5–1.4% fewer calls than the fixed threshold at a block's starting price and than a confidence-only gate (intervals exclude zero) and needs no second threshold; its accuracy differences are within noise, so we do not attribute the saving to the price. (Q2) The loss at L=128L=128 came from a window of a full block; with h=8h=8 or 16 every interval contains zero at 2.8–4.6% fewer calls (Table A28). On soundness: we no longer attribute the saving to the price gate (its edge over other gates is small; Tables A6 and A16), and every accuracy claim about release is a paired comparison with its interval.

W1. The confidence-only gate nearly matches the price gate (old Table 10, now Table A6). We agree, and this observation changed the revision. Since the window supplies the saving and any threshold test can serve as the gate, we now apply release, unchanged, to five settings of three admission rules and to AdaBlock-dLLM. Release applied unchanged (windows frozen at h=16h=16 for LLaDA and h=32h=32 for Dream; RTX4090, full test sets, paired) gives (Table 2, Table A18):

Admission ruleLLaDA: Δ\Delta calls / Δ\Delta acc [95% CI]Dream: Δ\Delta calls / Δ\Delta acc [95% CI]η\eta (LLaDA / Dream)
Margin price (σ=4\sigma=4), i.e. TAPER−6.1% / −0.19 [−1.04, +0.63]−8.2% / +0.17 [−0.62, +0.99]0.103 / 0.108
Fast-dLLM (τ=0.80\tau=0.80)−6.1% / −0.49 [−1.69, +0.71]−8.4% / −0.08 [−1.43, +1.20]0.082 / 0.094
Fast-dLLM (τ=0.90\tau=0.90)−5.1% / −0.67 [−1.53, +0.17]−6.2% / +0.19 [−0.65, +1.04]0.093 / 0.093
Fast-dLLM (τ=0.95\tau=0.95)−4.1% / −0.45 [−1.03, +0.10]−5.0% / +0.54 [−0.28, +1.38]0.100 / 0.092
Fast-dLLM factor−6.4% / −0.08 [−0.95, +0.77]−7.0% / +0.50 [−0.85, +1.82]0.102 / 0.087
AdaBlock-dLLM (adaptive blocks)−6.1% / −0.75 [−2.03, +0.51]−4.4% / +0.57 [−0.38, +1.56]0.097 / 0.073

The 87–93% recovery in the gate ablation is what a gate-agnostic step predicts. What the gate ablation shows beyond this is that dropping AHD's history condition is what matters: with admission, quota and window fixed, AHD's anchored-history gate removes only 0.1% (LLaDA) and 1.4% (Dream) of calls, against 6.1% and 8.2% for the price gate and 5.6% and 7.1% for confidence alone (Table A6). Two design properties make release more than a wider block: it shares the admission test, so it adds no threshold beyond the window h and plugs into a rule as it stands, without changing its threshold (the five settings and AdaBlock-dLLM above); and window commits never count toward the active block's quota, so the active block keeps its progress guarantee and the call budget Tmax⁡T_{\max} is unchanged (Proposition 1). The contributions are this release step, the exact ledger and the controls that locate its saving.

W2. Notation grafted onto Section 2. Section 2 ("Blocks and unified sets", "Threshold admission") now defines every set used later: the call index n≡(b,k)n \equiv (b,k), the active masks CnC_n, the window masks FnF_n, and the split of each call's commits into admissions and releases, An=AnC∪RnA_n = A_n^C \cup R_n (RnR_n is empty when h=0h=0). Admission is one threshold rule: each active position gets a risk rr and is committed if r<τkr < \tau_k. Fast-dLLM's threshold is the case r=1−p(1)r = 1-p(1), τk=1−τ\tau_k = 1-\tau, and our margin price the case r=1−(p(1)−p(2))r = 1-(p(1)-p(2)), τk=dk\tau_k = d_k. Section 3, Proposition 1 and Algorithm 1 then specify release as r<γτkr < \gamma\tau_k on FnF_n for either rule, and the ledger (Proposition 2) uses the same index, wn=∣AnC∣w_n = |A_n^C|. Table A1 lists every symbol.

AuthorsOpenReview ↗

Reply to Reviewer F9E1 (part 2 of 2)

Q1. Does the price gate outperform a tuned fixed threshold? In calls, slightly; in accuracy, not resolvably; and it has no second threshold to tune. With admission, quota and window fixed, we swept fixed release thresholds over the price's range, 0.125, 0.1875 and 0.25 (Table A16). Against 0.125, the price at a block's first call, the scheduled price uses 1.1% [0.7, 1.4] (LLaDA) and 1.4% [1.1, 1.7] (Dream) fewer calls at +0.18 [−0.19, +0.62] and +0.35 [0.00, +0.71] points. The two higher thresholds save 0.5–3.6% more calls and are less accurate, a trade-off (on LLaDA by 0.54 [0.03, 1.10] and 0.83 [0.08, 1.66] points). Against a confidence-only gate the price gate uses fewer calls in all three pairings (0.5% [0.1, 0.7], 1.2% [0.8, 1.5] and 1.0% [0.6, 1.4]); the accuracy differences, 0.14–0.33 points in its favour, have intervals containing zero (Table A6, Appendix C.1). These margins are small next to the window's 5–8%, so the paper presents release as a step for any threshold rule rather than crediting the price. "Priced" in the title means that release is judged at the admission rule's own threshold, its price at that call. For admission, a constant price σ/H\sigma/H lies on the same accuracy–calls curve as the rising price (Table A29).

Q2. Lower accuracy at short lengths (old Figure 3, now Figure A1). The loss came from using a full-block window on a four-block output; a window of h=8h=8 or 16 removes it (every paired interval contains zero) at 2.8–4.6% fewer calls. In detail: at L=128L=128 (four blocks of 32) the common window h=32h=32 is a full block, a quarter of the output. On the three datasets of Figure A1, accuracy falls by 0.92 (LLaDA; the interval excludes zero) and 1.01 (Dream) points; with GSM8K added, by 0.53 and 0.40 points, with intervals containing zero (Table A15). On Dream, short outputs also remove most of the end-of-sequence padding that release commits at L=256L=256 (10.8 padding positions per output at L=128L=128 against 69.9), so at least 54% of its early commits become answer content committed a block ahead (Appendix C.3). The remedy is a window proportional to the output. With every window paired on the same items (new Table A28, full four test sets):

Window (L=128L=128)LLaDA acc / callsLLaDA Δ\Delta acc [95% CI] / Δ\Delta callsDream acc / callsDream Δ\Delta acc [95% CI] / Δ\Delta callsη\eta (LLaDA / Dream)
h=0h=0 (no release)43.66 / 42.12reference54.74 / 56.07referencen/a
h=8h=843.69 / 40.73+0.03 [−0.59, +0.60] / −3.3%54.51 / 54.48−0.23 [−0.82, +0.35] / −2.8%0.122 / 0.164
h=16h=1643.90 / 40.17+0.24 [−0.28, +0.75] / −4.6%54.43 / 53.93−0.31 [−0.98, +0.32] / −3.8%0.100 / 0.145
h=32h=3243.36 / 38.22−0.30 [−1.13, +0.48] / −9.3%54.34 / 52.83−0.40 [−1.73, +0.92] / −5.8%0.131 / 0.159

Every interval contains zero. On LLaDA, h=16h=16 is +0.54 [−0.15, +1.25] points above h=32h=32, and one early commit removes 0.10–0.16 calls at L=128L=128, against 0.086–0.121 at L=256L=256 (Section 4.7). Across L=128L=128, 256 and 512 with B=32B=32, every window up to h=L/8h = L/8 kept the paired interval containing zero; the one exclusion is the full-block window h=32h=32 = L/4 at L=128L=128 on three datasets. This is a description of the measured windows, not a rule we validated.

AuthorsOpenReview ↗

Note to Reviewer F9E1

Thank you again for your review. Following our response of 12 November, the attached revision makes the distinctions behind the gate ablations and short-output controls more explicit.

  • W1, the confidence-only gate. We agree that the price gate is not essential to obtaining most of the saving. Early release reduces calls across five settings of three admission rules in the uncached L=256L=256 comparison. The paired accuracy intervals for these settings include zero (Tables 2 and A18), which does not establish equal accuracy.
  • W2, notation. The notation runs consistently from Section 2 through Algorithm 1, with definitions collected in Table A1.
  • Q1, price gate or fixed threshold. Reusing the active block's margin price avoids a separate release threshold. It also uses slightly fewer calls than the tested confidence-only gate or a threshold fixed at the block's starting price. Their paired accuracy intervals include zero (Tables A6 and A16). Higher fixed thresholds save more calls but have lower accuracy point estimates, with significant losses on LLaDA (Table A16).
  • Q2, short outputs. The original loss occurred with a full-block window at L=128L=128. The shorter-window controls, h=8h=8 or h=16h=16, reduce calls by 2.8–4.6%, and their paired accuracy intervals include zero (Table A28). The window therefore needs to suit the output length.

Figure 2 now labels settings directly at full text width. Tables 2 and 6 show early release off and on, and their paired intervals are retained in Table A18. Table 2's cache row shows the overall cached comparison, and Table A23 retains the equal-query-width control. Table numbering and the underlying results are preserved. If a concern from our earlier response remains open, we would be glad to address it.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response (to all reviewers)

We thank the reviewers. The revised PDF marks every change in blue. All new runs use one GPU type (RTX4090) and are paired on identical prompts.

What the paper claims, in four sentences. 1. Release commits low-risk positions in a short window after the active block, judged by the admission rule's own test. It adds no model call and no threshold, and the block keeps its progress guarantee. 2. It works on any threshold rule: on three admission rules at five settings (our margin price, Fast-dLLM's threshold at three values, Fast-dLLM's factor), on AdaBlock-dLLM's adaptive blocks and on three models it removes 4–8% of calls, and every paired accuracy interval contains zero (Tables 2 and A18). 3. Added to a decoder that already uses Fast-dLLM's dual key–value cache, it lowers wall-clock time by 4–14%: a removed call saves a cached call, and a block that is filled before it opens skips its full-forward refresh (Tables 6 and A26). 4. An exact ledger explains the size of the saving: one early commit removes only about 0.1 calls, so early-commit methods should report the calls saved per early commit.

What changed in the paper.

  • The abstract, introduction and Sections 2–6 are rewritten; Section 4 follows four questions; Sections 2 and 3 share one notation (Table A1).
  • The main text now holds the comparison with admission-and-release schedulers (Table 3), terminal shares for every window (Table 4), tokens per second for 14 decoders (Table 5) and the key–value cache (Table 6).
  • Figure 2's L = 512 row now shows every decoder: Fast-dLLM's rules with and without release, ADAS, KLASS and LESS.
  • New experiments, each with the request it answers: release on Fast-dLLM's rules (F9E1 W1, cnUk Q2), on AdaBlock-dLLM (Y69P Q1) and on LLaDA-1.5; AHD and D2F (Y69P W2, Q1) and DUS and SlowFast (cnUk Q3) in our pipeline; a key–value cache with an equal-query-width control and a cost ledger (cnUk Q1) and retuned prices (cnUk Q2); windows at L = 128 (F9E1 Q2); window selection on development problems alone (cnUk W4).
  • The appendix is regrouped into seven sections (A–G) and opens with a contents page; the experiments added during the discussion are in Appendix F (F.1–F.4). Tables that repeated other tables are merged, and several tables show LLaDA and Dream side by side.

Results against us, stated in the paper. AdaBlock-dLLM is the most accurate LLaDA setting. At L = 512, KLASS and LESS are more accurate than TAPER on Dream (at 1.6–2.1 times its calls), Fast-dLLM factor alone beats TAPER there, and on LLaDA's factor rule release costs 1.35 points. Without a cache, retuning the margin price matches release on LLaDA. The L = 512 gain over retuning of the submission did not replicate on development problems and is no longer claimed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.