acceptodds
Under review as a conference paper at ICLR 2027

One Block Ahead: Promoting Sampler State in Diffusion Language Models

Abstract

Block-diffusion language models denoise a whole block of token positions in parallel so each forward pass can emit many tokens. However, blocks are still denoised in strict sequence, a cost that within-block acceleration leaves untouched. Here, we investigate decoding blocks concurrently and propose One Block Ahead (OBA), a training-free decoding algorithm that denoises the next block alongside the current one. We demonstrate that for uniform-state diffusion LMs, where a block’s denoising progress is not visible in its intermediate tokens, concurrent decoding is advantageous if the next block’s tokens are promoted together with the corresponding sampler state, including the self-conditioning signal and the denoising time. We also derive a break-even criterion that relates the benefit of OBA to the relative cost of a paired-block forward pass. On DiffusionGemma, a state-of-the-art diffusion LM, OBA increases the tokens per forward pass by 42.3% (from 15.64 to 22.25) and the single-request denoising throughput by 16.3%, averaged over ten benchmarks, with a small accuracy regression.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language models, block diffusion, uniform-state diffusion, parallel decoding, multi-block decoding, inference efficiency, training-free acceleration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SfW1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes One Block Ahead (OBA), a training-free inter-block parallel decoding method designed to further improve the inference efficiency of diffusion language models. The key idea is to start denoising the next block before the current block is completed, reusing the accumulated progress through sampler-state inheritance, while employing block-causal attention and optimized GPU kernels to reduce computational overhead. Experiments on DiffusionGemma demonstrate promising efficiency gains, with a 42.3% increase in tokens per forward pass and a 16.3% improvement in throughput, at the cost of a 1.37-percentage-point drop in average accuracy. Overall, the method primarily improves inference efficiency rather than generation quality.

Critical strengths

I think the main strengths of this paper mainly show from the experiment perspective.

  1. The paper presents a solid and extensive experimental evaluation across multiple benchmarks. Through detailed ablation studies, the authors systematically investigate different aspects of OBA, including sampler-state inheritance, concurrent decoding schedules, attention mechanisms, and kernel-level optimizations. These analyses not only validate the effectiveness of the proposed method but also provide valuable insights into the roles of individual components and how they contribute to overall inference efficiency.
  2. The paper demonstrates meaningful engineering improvements by further accelerating DiffusionGemma, which already achieves high inference efficiency, through concurrent inter-block decoding and optimized GPU kernels. The reported 42.3% increase in tokens per forward pass and 16.3% improvement in throughput, without requiring additional training, highlight the practical value of the proposed approach for efficient diffusion language model inference.
Critical weaknesses

The main weakness which makes me confused is the limited comparison with FlowBlock: OBA's inter-block parallel decoding mechanism is highly similar to the existing FlowBlock approach. The main differences lie in the warm-up initiation policy and the sampler-state management and computational optimizations tailored to DiffusionGemma. However, the sampler-state components are inherited from DiffusionGemma, and the adapted FlowBlock baseline employs the same state inheritance mechanism. Although the authors make substantial efforts to adapt FlowBlock to DiffusionGemma, they retain the original readiness thresholds without systematic tuning for the new sampling mechanism. Since the warm-up initiation time is the key distinction between the two methods, the current experiments do not sufficiently establish OBA's advantage over a properly tuned FlowBlock baseline or its independent algorithmic novelty. A sensitivity analysis of the readiness threshold and a fair comparison of the accuracy–throughput trade-off would strengthen the evaluation.

Questions for authors

I raised four key questions for authors, and helpful answers will contribute to my rate update.

  1. Although the authors adapt FlowBlock to DiffusionGemma, they retain the original readiness thresholds (0.6 and 0.8). Given the differences in sampling mechanisms, block sizes, and readiness definitions, could the authors provide a sensitivity analysis of these thresholds and compare the resulting accuracy–throughput trade-offs? In particular, does OBA still maintain a clear advantage when FlowBlock is allowed to initiate warm-up earlier?
  2. The adapted FlowBlock and OBA share the same sampler-state inheritance mechanism, block-causal attention, and paired-block kernels, with the main difference being the warm-up initiation time. Could the authors clarify what fundamentally new algorithmic mechanisms OBA introduces beyond existing approaches? In particular, would FlowBlock become equivalent to OBA if its readiness gate were configured to allow immediate warm-up?
  3. The experiments show considerable variation in warm-up effectiveness across benchmarks, with relatively lower early token acceptance rates on tasks such as AIME and LiveCodeBench. Could the authors further investigate whether these differences are related to cross-block contextual dependencies or prediction stability? Is a fixed immediate warm-up strategy suitable for all generation scenarios, or could an adaptive strategy based on online sampling states achieve a better accuracy–throughput trade-off?
  4. OBA is evaluated only on DiffusionGemma, and its key designs are closely tied to uniform-state diffusion and model-specific sampler states. Could the authors clarify which DLM architectures or sampling paradigms OBA can generalize to, and what technical conditions are required for its applicability? If extending OBA to other models is challenging, it would be valuable to explain the underlying limitations and distinguish generalizable components from DiffusionGemma-specific adaptations. Even without additional cross-model experiments, a clearer discussion of these applicability boundaries would help the community better understand the broader significance of the work.

PS:The paper contains a few incomplete citation placeholders (?), and some statements in the Background and Related Work sections are overly absolute. For example, the claim that fully bidirectional DLMs cannot reuse KV caches overlooks existing approximate caching methods. The authors should address these citation issues and refine the technical statements for greater accuracy.

2 comments

1. Readiness-threshold sensitivity and accuracy–efficiency trade-offs: We agree that the transferred thresholds of 0.6 and 0.8 need not be optimal for DiffusionGemma. We also compare Native and OBA across the same temperature schedules: 0.8→0.4, 0.7→0.2, and 0.85→0.5. All comparisons use matched prompts, per-example seeds, output limits, and evaluation procedures.

In the sweep, immediate OBA achieves 83.80% accuracy, 22.30 TPF, and 2,766 denoising TPS. FlowBlock at readiness 0.2 achieves 83.10%, 20.87 TPF, and 2,839 TPS; readiness 0.4 achieves 84.35%, 18.33 TPF, and 2,641 TPS. Earlier admission is therefore competitive, with different policies favoring different accuracy–throughput trade-offs. Immediate OBA retains the highest TPF among these policies without readiness-threshold tuning. All three use full sampler-state promotion, the mechanism enabling useful inter-block progress in this uniform-state setting.

2. Relationship to FlowBlock and the contribution of OBA: In our adaptation of FlowBlock to uniform-state diffusion, immediate admission is indeed equivalent to OBA. However, this adaptation is not addressed in the original FlowBlock [1]. We study a setting combining strong reasoning performance, 256-token blocks, and an already efficient baseline of approximately 16 tokens per forward, extending beyond the settings evaluated in prior inter-block decoding work. Our contributions are identifying which sampler states must be promoted to preserve useful warm-up progress, and providing a comprehensive component-wise analysis of how state promotion, admission timing, attention structure, and kernel optimizations affect multi-block decoding. We will clarify this positioning in the revision.

3. Task-dependent warm-up effectiveness and adaptive admission: To answer the question, we added an analysis using all 30 questions from each AIME benchmark and length-stratified samples of 30 prompts each from LiveCodeBench and GSM8K. We measure consecutive warm-up prediction changes and sensitivity to predecessor-state updates while holding the warm-up sampler state fixed.

Rebuttal Table 1. Warm-up prediction stability and sensitivity to predecessor updates

DatasetWarm-up prediction changesPredecessor changes
AIME 202437.9%38.1%
AIME 202537.9%34.5%
LiveCodeBench30.0%27.3%
GSM8K30.5%29.0%

As shown in Rebuttal Table 1, the experiment results confirm sensitivity to preceding-block updates, but do not establish consistently greater instability or sensitivity on AIME and LiveCodeBench than on GSM8K. Crucially, the paper’s full evaluation shows that OBA raises first-current-pass acceptance on AIME and LiveCodeBench from 3.4–3.9% to 25.9–30.7% (Figure 2), demonstrating useful warm-up progress despite evolving context.

Our readiness sweep also shows that state-dependent admission can improve the trade-off on individual tasks. On LiveCodeBench, readiness 0.2 improves accuracy from 64.10% to 65.86% and denoising throughput from 2,224 to 2,454 TPS relative to immediate admission. Because admission depends on the evolving sampling state, this directly tests an online-state-based alternative. No evaluated threshold is consistently best across benchmarks: immediate warm-up is a useful default, while admission timing provides an additional task-dependent control.

4. Applicability beyond DiffusionGemma: Thank you; we will add this discussion to the paper. OBA’s general principle is that a promoted block must carry whatever sampler state its tokens cannot reveal. In masked diffusion, the mask shows progress, so the tokens suffice (as in D2F and FlowBlock). In uniform-state diffusion, the tokens do not show how far a block has progressed, so the self-conditioning signal and denoising time must be promoted too. Continuous diffusion (e.g., SSD-LM) is the closest analogue: the state to promote is the noisy latent, its noise level and any self-conditioning. OBA requires block-autoregressive decoding and a decoder that accepts a block-causal mask over two blocks. The paired-block layout, full-state promotion and break-even analysis are general; the state contents, kernels and measured speedups are specific to DiffusionGemma, which is, to our knowledge, the only public large-scale model that is both uniform-state and block-autoregressive.

PS. Citations and technical statements: We thank the reviewer for catching these. We will correct the incomplete citations, which should cite Fast-dLLM v2 [2]. We will also narrow the statements the reviewer points to and cite approximate caching such as Fast-dLLM [3], which Section 5 also discusses.

[1] https://arxiv.org/abs/2607.17652

[2] https://arxiv.org/abs/2509.26328

[3] https://arxiv.org/abs/2505.22618

Reviewer_SfW1OpenReview ↗

Response to Rebuttal

Thank the authors for their detailed rebuttal and additional experiments. However, the response does not sufficiently distinguish OBA from FlowBlock and instead reformulate the contribution primarily as an engineering adaptation and system-level optimization. While these efforts have practical value, my concerns about independent methodological novelty remain unresolved and therefore I will maintain my original score.

Reviewer_1bqaOpenReview ↗
Rating
1
Confidence
3(−1)
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes One Block Ahead (OBA), a training-free decoding algorithm for accelerating block-diffusion language models through concurrent denoising of adjacent blocks. OBA employs block-causal attention to isolate the current block from its successor and promotes the successor's tokens, self-conditioning signal, and denoising time upon commitment. The central insight is that preserving sampler state is essential for exploiting inter-block parallelism in uniform-state diffusion models. The paper also derives an approximate throughput break-even criterion. On DiffusionGemma, OBA improves tokens per forward pass by 42.3% and denoising throughput by 16.3%, with a 1.37-percentage-point reduction in average accuracy.

Critical strengths

See weakness

Critical weaknesses

Section 7 (limitations) exceeds the page limit. Therefore, it should be desk-rejected.

Questions for authors

See weakness

2 comments

We thank the reviewer for pointing this out. The Limitations paragraph (five lines) does start at the top of page 10. We numbered it as a section by mistake: it was meant as a short end-of-paper note, like the statements that follow it. The method, experiments and conclusion end on page 9, and the paragraph adds no result or claim. We will revise the paper accordingly to avoid this confusion.

We would welcome the reviewer's assessment of the technical content: the role of sampler-state promotion in uniform-state diffusion (Table 2), the break-even criterion α > r − 1 (Section 3.3), and the efficiency results (Table 1). We are glad to answer any questions during the discussion.

Thanks for your rebuttal.

Based on the review of the paper and observing the interaction with the other reviewers, I maintain my original score of 1. This is the decision after considering the technical aspects of this paper. The reasons include:

  • The limited novelty over FlowBlock,
  • Limited model coverage, restricted to DiffusionGemma,
  • Failure to comply with the page limit policy in the pre-rebuttal version.
Reviewer_knx6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes One Block Ahead, a training-free decoder for DiffusionGemma, a block-autoregressive, uniform-state diffusion LM. Each pass denoises the current 256-token block together with the next block under a block-causal mask, and when the current block commits, the warm-up block is promoted with its tokens, self-conditioning signal and denoising time. On ten benchmarks at batch size 1 on one GB300, OBA raises tokens per forward from 15.64 to 22.25 and denoising throughput by 16.3%.

Critical strengths
  1. The state-promotion finding is clean and useful. Table 2 shows that promoting only the tokens doesn't pay off (alpha = 0.25, below r - 1 = 0.26, TPS -2.8%), adding the self-conditioning signal and then the denoising time brings alpha to 0.41 and 0.54 (TPS +16.3%), and withholding self-conditioning during warm-up makes the extra pass nearly useless. That's a non-obvious point about uniform-state models.
  2. The measurements are careful: matched prompts and seeds, three seeds for the main comparison, per-shape kernels in one engine, and wall-clock TPS alongside denoising TPS.
Critical weaknesses

Points 1 and 2 drive my score.

  1. It's a single operating point, so it doesn't show a better accuracy/throughput trade-off. OBA loses 1.37 points on average and drops on 9 of 10 benchmarks, by 2.5-3.3 points on GPQA, IFEval, LiveCodeBench and AIME 2024 (Table 6), and it's still -1.56 without the two AIME sets. DiffusionGemma already has knobs that trade accuracy for speed (entropy budget, stopping threshold, step count), and the ablations suggest the speedup and the accuracy loss come from the same place (No Accept keeps 84.95 accuracy but gains only 1.2% TPS). OBA is also basically FlowBlock with its readiness gate at zero, and FlowBlock keeps 0.73 more points. Without an iso-accuracy comparison or a sweep, the 16.3% may just be a different point on the same curve.
  2. The evidence is narrow. One model, batch size 1, one GPU. The gain relies on spare compute at 256 positions (r = 1.26), which will likely shrink with batching. The break-even criterion is bookkeeping rather than a prediction, since alpha is fitted after running OBA (Appendix D calls the TPS relation an identity).
  3. The argument against wider blocks rests on a confounded cost. A dense 512-position pass costs 1.70-1.84x vs 1.26x for OBA, which the paper attributes to the mask. But both halves attend to the same long cache, the MoE/dense work is unchanged, and OBA has its own fused kernels. With Dense Mask's alpha = 0.38, a cost near 1.26x would mean a speedup, not -20.9%. The two citations supporting the "compute-bound at 256 positions" claim are broken ("(?)", p.2 and p.6).
  4. Positioning. Staggered or pipelined denoising across positions is an old idea outside block diffusion, e.g. AR-Diffusion (NeurIPS 2023) for text and Diffusion Forcing, Rolling Diffusion and FIFO-Diffusion for sequences and video. Within diffusion LMs, AsyncLane (2026) also starts the next segment before the previous one commits. None of these are discussed, so the paper should say more clearly that its novelty is promoting the sampler state in a uniform-state model, not concurrent decoding itself.
  5. Smaller things. The accuracy differences have no CIs (my rough unpaired standard error is about 1 point), the Table 2 ablations are single-seed, and FlowBlock's thresholds are reused from LLaDA2.1 without retuning.
Questions for authors
  1. What throughput does Native (C=256) reach when its entropy budget, stopping threshold or step count is tuned to OBA's accuracy, and what accuracy does OBA reach at a stricter budget? A FlowBlock threshold sweep would also help.
  2. Can you give paired CIs for the accuracy differences in Table 6?
  3. What's the per-component time (attention vs. MoE/dense) of a pass under OBA, Dense Mask and Native (C=512)?
3 comments

We thank the reviewer for recognizing our state-promotion finding as clean and useful. Our central contribution is identifying what makes inter-block parallelism effective in a uniform-state diffusion LM: promoting tokens alone yields −2.8% throughput, whereas preserving tokens, self-conditioning, and denoising time yields +16.3%. The additional experiments below examine whether this benefit extends beyond the original sampler setting and explain the measured computational advantage.

1. Accuracy–efficiency trade-offs of Native, OBA, and FlowBlock: We now report both accuracy–TPF and accuracy–TPS trade-off for Native and OBA under three temperature schedules of DiffusionGemma: 0.8→0.4, 0.7→0.2, and 0.85→0.5, averaging three seeds across all ten benchmarks. We vary temperature because our preliminary tests found it the most effective sampler control for varying TPF; entropy budget and stopping thresholds remain fixed.

The most relevant comparison is between Native at temperature schedule 0.7→0.2 and OBA at default schedule. Native achieves 82.86% accuracy, 17.45 TPF, and 2,682 TPS, while OBA achieves 82.70%, 22.25 TPF, and 2,761 TPS. At iso-accuracy (a 0.16-point difference) OBA retains 27.5% higher TPF and 2.9% higher denoising throughput. While there’s no noticeable TPS gain when comparing the two operation points, we would like to highlight that, if given a GPU setup with more compute capacity, more of the TPF gain can be transferred to TPS gain.

Rebuttal Table 1. Temperature sweep. Unweighted means across all ten benchmarks, including AIME, over three seeds. TPS denotes denoising throughput. The 0.8→0.4 rows reproduce the original paper results.

MethodTemperatureAccuracy (%)TPFTPS
Native0.85→0.585.9214.822,249
Native0.8→0.484.0715.642,375
Native0.7→0.282.8617.452,682
OBA0.85→0.582.6320.352,497
OBA0.8→0.482.7022.252,761
OBA0.7→0.279.9524.573,080

We also sweep the readiness gate of our adapted FlowBlock to uniform-state diffusion language model. We gave OBA's state promotion and paired-block kernels so that the comparison isolates the start time (Section 4.1). The original FlowBlock as published in [1], where a block carries only its tokens: with that rule an immediate start is 2.8% slower than Native (Keep Tokens, Section 4.3). The early start pays only once the self-conditioning signal and denoising time are promoted too.

Rebuttal Table 2. FlowBlock readiness sweep. Unweighted means over the ten benchmarks at default temperature schedule. Native, OBA and the mixed policy (0.6 for general and math, 0.8 for code) are the paper's results; readiness gate 0.2 and 0.4 are new.

MethodAdmissionAccuracy (%)TPFTPS
Native (C=256)None84.0715.642,375
OBAImmediate82.7022.252,761
FlowBlockReadiness 0.283.1020.872,839
FlowBlockReadiness 0.484.3518.332,641
FlowBlockMixed 0.6 / 0.883.4316.762,482

With the promoted state, every new gate lands within 5% of OBA's TPS, and OBA has the highest TPF with no threshold to tune. A gate at 0.2 trades 6.2% of TPF for 2.8% more TPS, since fewer of its passes are paired; a gate at 0.4 is 4.3% slower. The accuracy differences from OBA (0.40 and 1.65 points) are inside the 95% interval for OBA − FlowBlock, [−2.87, +1.42] points. So the start time is a second-order control, and state promotion is the first-order effect, which is the paper's claim. Every policy in Rebuttal Table 2 carries the promoted state and is faster than Native; with token-only promotion, the same immediate start is slower than Native.

We will revise Sections 4.2 and 5 to say this precisely: the margin over FlowBlock at its published thresholds comes from the start time, and across gates the gain comes from state promotion.

2. Evaluation scope and purpose of the break-even analysis: Our evaluation targets batch-size-one, latency-sensitive decoding, where exploiting compute capacity within a single request is directly useful. Larger-batch scaling is a separate question, and our claims are scoped to the evaluated setting.

The break-even analysis makes the state-promotion result interpretable: token-only promotion provides insufficient reusable progress to offset the paired-pass cost, while full-state promotion crosses this threshold and delivers a throughput gain. Its purpose is to explain which design choices make concurrent decoding worthwhile. We will make explicit that this is an explanatory criterion, rather than a predictor for unseen workloads.

3. Why 512-position passes cost more: The added component profiling identifies where the measured cost difference arises. At context lengths of 2,048 and 16,384, OBA’s attention kernel work is approximately 1.09 / 2.62 ms, compared with 3.27 / 11.40 ms for Dense Mask, while MoE and dense/projection costs remain similar. Native C512 closely tracks Dense Mask. The gap is in the attention kernel: OBA's offset-aware paired attention skips the quadrant that the block-causal mask removes, and a dense input has no such quadrant. Overall, OBA’s pass latency is 19.7–45.6% lower than Dense Mask’s across the profiled settings. We will also correct the incomplete citations, which should cite Fast-dLLM v2 [2].

4. Positioning and contribution: Concurrent and staggered denoising have substantial prior work, which we will discuss more fully. To our knowledge, OBA is the first study of inter-block pipelining for a uniform-state diffusion LM, targeting a model with strong reasoning performance, approximately 16 tokens per forward, and 256-token blocks.

The substantive finding is how to preserve useful warm-up computation in this setting: tokens, self-conditioning, and denoising time jointly determine whether concurrent decoding produces a net benefit. We deliberately equip the adapted FlowBlock baseline with the same state promotion and paired-block kernels so that the comparison isolates admission timing. This shared implementation enables a fair scheduling comparison; the state-promotion ablations establish the mechanism that makes both schedules effective on DiffusionGemma.

5. Accuracy uncertainty and additional ablation seeds: Using the original Table 6 runs, the paired 95% confidence intervals across three matched generation seeds are [−2.79, +0.07] percentage points for OBA−Native and [−2.87, +1.42] for OBA−FlowBlock. We also provide per-dataset intervals, computed using Student-t intervals over matched seed differences.

Rebuttal Table 3. Original Table 6 accuracy and paired confidence intervals Confidence intervals are in percentage points.

DatasetNativeFlowBlockOBAOBA − NativeOBA − FlowBlock
GSM8K96.3496.0396.06[−1.69, +1.14][−0.55, +0.60]
MATH-50096.1395.8095.20[−6.03, +4.17][−5.07, +3.87]
MBPP94.8794.8793.80[−2.10, −0.03][−5.08, +2.95]
HumanEval97.3696.7596.75[−6.67, +5.45][−4.01, +4.01]
GPQA71.5570.5469.02[−6.29, +1.24][−9.74, +6.71]
IFEval94.0992.6791.44[−6.33, +1.03][−3.54, +1.08]
MMLU88.1387.8386.90[−2.75, +0.28][−3.68, +1.81]
LiveCodeBench C++66.6762.0463.44[−6.04, −0.42][−1.19, +3.98]
AIME 202474.4474.4471.11[−25.24, +18.57][−19.89, +13.23]
AIME 202561.1163.3363.33[−16.90, +21.35][−24.84, +24.84]
Average84.0783.4382.70[−2.79, +0.07][−2.87, +1.42]

The expanded ablations also clarify the reviewer’s observation on No Accept. Across three seeds, No Accept averages 83.48%, compared with 84.28% for Keep Signal. Both retain only self-conditioning at promotion, but Keep Signal permits acceptance during warm-up. No Accept’s original advantage over this more closely matched control therefore reverses in the three-seed average. We will revise the paper to show the three-seed result does not establish that early acceptance causes the reported accuracy loss.

[1] https://arxiv.org/abs/2607.17652

[2] https://arxiv.org/abs/2509.26328

Thanks for the additional experiments. The profiling and the extra runs address some of my earlier concerns. I still share Reviewer SfW1’s concern about what OBA adds beyond FlowBlock. The new results make a good case for keeping the sampler state, but it’s less clear that starting immediately is better than using a tuned gate. Do you have end-to-end throughput numbers for OBA and the tuned native baseline at similar accuracy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.