acceptodds
Under review as a conference paper at ICLR 2027

Large Flow Language Models

16 views

Abstract

Flow and diffusion models are leading approaches to generative modeling of continuous signals such as images, videos, audio, and protein structures. Their parallel generation makes them attractive for fast language modeling, but it conflicts with the inherent directionality of text where earlier tokens contextualize the later ones, whereas conventional flows denoise all positions synchronously. As a consequence, flow language models have largely remained at sub-billion scale and have not established competitive quality-speed tradeoffs on realistic tasks. We present Lafa, a block-autoregressive flow language model that preserves causal order across blocks while learning an asynchronous progression within each block, and scale it to 8B parameters by converting a pretrained autoregressive model using less than 10B training tokens. Lafa retains both a block denoiser and a next-token predictor, unifying sequential next-token prediction, parallel flow decoding, confidence decoding, and self-speculative decoding, in a single model. Across six benchmarks spanning instruction following, reasoning, and code, these modes trace a broad quality-speed Pareto frontier, and Lafa attains 2.54x the throughput of a strong autoregressive baseline at matched quality. Our findings show that continuous flows can be a practical and scalable foundation for language modeling.

Primary area:
generative models
Keywords:
flow language models, diffusion models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GUamOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Lafa converts pretrained autoregressive language models into block-autoregressive flow models while retaining next-token prediction. It combines bidirectional attention within noisy blocks, learned tokenwise time schedules, and joint autoregressive and denoising objectives. The resulting model supports iterative flow decoding, distilled flow-map decoding, confidence decoding, and self-speculative decoding.

Experiments cover models from 0.5B to 8B parameters, adapted using approximately 9B tokens. The strongest speed result comes from one-step block drafting with fused self-verification: Lafa reports 2.54× the decoding throughput of the matched AR baseline at comparable average quality on three tasks.

Critical strengths
  • The paper provides a concrete recipe for scaling continuous flow language models. Converting pretrained AR checkpoints to 7–8B flow models while retaining next-token prediction is a useful result. The blockwise formulation also preserves prefix caching, making the approach practical to implement.

  • The self-speculative results show a meaningful speed–quality tradeoff. Table 32 reports actual decoding throughput rather than relying only on tokens per forward. At B=8, Lafa reaches 93.8 tokens/s with an average score of 76.3, compared with 39.5 tokens/s and 73.9 for matched AR SFT. Comparisons with MTP, MARS, and diffusion models strengthen the evidence beyond the AR baseline.

Critical weaknesses
  • The ablations do not establish which flow-specific designs drive the main acceleration result. Table 2 evaluates training choices at 0.5B using 32-step flow decoding, whereas the headline speedup uses a 7B model with one-step drafting and self-verification. Improvements in iterative flow quality do not directly establish improvements in draft acceptance or decoding speed. This leaves the contribution of asynchronous schedules and the flow-training recipe to the strongest practical result unresolved.

  • The throughput comparison leaves questions about implementation fairness. Appendix E.5 uses standard cached Hugging Face generation for AR and a specialized fused verification-and-redrafting implementation for Lafa. It also disables block caching for Fast-dLLM v2 without clearly explaining this choice. These settings do not invalidate the measurements, but they make it difficult to separate the method’s advantage from differences in inference optimization. Since speed is central to the paper, this materially affects my assessment.

Questions for authors
  • Contribution clarification: Do existing experiments evaluate synchronous schedules or a simpler block-prediction objective under the same fused self-speculative decoder? Acceptance rates and wall-clock results at 7B would help establish which flow-specific choices contribute to the reported speedup.

  • Evaluation clarification: Why was block caching disabled for Fast-dLLM v2, and what inference optimizations were used for the AR and parallel-generation baselines? Please clarify whether the reported advantage holds under comparable optimization settings.

4 comments

We thank the reviewer for the positive assessment, particularly for recognizing the significance of scaling continuous flow language models and self-speculative speed–quality results. We address the two questions below.

W1 Q1. The ablations do not establish which flow-specific designs drive the main acceleration result [...] This leaves the contribution of asynchronous schedules and the flow-training recipe to the strongest practical result unresolved. / Do existing experiments evaluate synchronous schedules or a simpler block-prediction objective under the same fused self-speculative decoder? Acceptance rates and wall-clock results at 7B would help establish which flow-specific choices contribute to the reported speedup.

We first clarify that a direct comparison between training variants is less straightforward in self-speculative decoding than in flow decoding. In flow decoding, we can compare the variants in terms of accuracy at a fixed NFE. Yet, in self-speculative decoding, it is difficult to fix either speed or accuracy for a fair comparison. The accuracy is the same with the next-token prediction performance, while throughput depends on the acceptance rate of speculative tokens. Both come from the same weights, so a single training choice changes both the flow (drafter) and the next-token prediction (verifier). Each ablation checkpoint therefore differs in accuracy and in acceptance at the same time, and different training choices lead to different trade-offs between speed and accuracy. Nevertheless, in Table A we show the results of the 0.5B ablations of Table 2 under self-speculative decoding, including the synchronous (blockwise) schedules that the reviewer requested. We could not retrain the 7B model with these ablations within the rebuttal period, and we will try our best to add these results to the camera-ready version.

Table A: 0.5B self-speculative decoding ablation. Speedup is relative to AR decoding, and accuracy is averaged over GSM8K, HumanEval, and IFEval.

AblationTokens/ForwardAcceptanceTokens/Second (Throughput)SpeedupAvg. Acc. (%)
Blockwise uniform time schedule2.4356.11%78.192.44×45.03
Blockwise learned time schedule2.5058.67%80.842.53×43.67
Tokenwise learned time schedule (ours)2.5459.16%81.882.56×42.69
Frozen embeddings2.5960.16%82.912.55×41.76
Unconstrained embeddings2.6160.26%83.692.60×45.59
Norm-preserving embeddings (ours)2.5459.16%81.882.56×42.69
Flow CE + denoising MSE + AR (Ours)2.5459.16%81.882.56×42.69
Flow CE only3.7893.60%121.453.78×5.42
Denoising MSE only3.8094.01%119.683.74×5.66
Flow CE + denoising MSE3.7893.81%123.423.84×5.30

The last three rows show a clear example of the difficulty described above. Since they are trained without the AR loss, their accuracy collapses to about 5%, yet they accept about 94% of the drafted tokens and reach the highest speedup.

Among the variants with the AR loss, the tokenwise learned schedule gives the highest acceptance and throughput among the time schedules (2.43, 2.50, and 2.54 tokens/forward), so the asynchronous schedule contributes to the speedup over the synchronous schedule. For the embeddings, the three variants are close in throughput (2.55× to 2.60×), and the unconstrained variant is slightly ahead at 0.5B. Our final recipe thus remains close to the best in throughput while preserving accuracy in this setting, and it gives clear gains in flow mode (Table 2). We therefore believe that our design choices are reasonable for a single model that performs well under all decoding modes.

For a comparison on a simpler block-prediction objective, Table 7 in the appendix already provides the closest comparison at 7B. MARS is trained on the same data to predict a block from masked tokens, and we count its forward passes in the same way as for fused decoding in Lafa. At B=8, Lafa commits 4.05 tokens/forward, whereas MARS commits 2.99 at comparable accuracy.

W2 Q2. The throughput comparison leaves questions about implementation fairness. Appendix E.5 uses standard cached Hugging Face generation for AR and a specialized fused verification-and-redrafting implementation for Lafa [...] Since speed is central to the paper, this materially affects my assessment. / Why was block caching disabled for Fast-dLLM v2, and what inference optimizations were used for the AR and parallel-generation baselines? Please clarify whether the reported advantage holds under comparable optimization settings.

We appreciate the reviewer’s comment. We would like to first clarify that the AR baseline uses both KV caching and a highly optimized implementation offered by FlashAttention kernels, which is an established optimized configuration for AR inference [1] and is baked into Hugging Face as a standard [2]. We believe this optimization makes the comparison fairer to the AR baseline.

Second, we would like to emphasize that the fused verification and redrafting is a core part of the proposed method and is not merely an implementation choice that can be applied to the AR baseline. This decoding algorithm requires a model that performs both next-token prediction and block drafting simultaneously, so a plain AR model cannot use it (Section 3.3, Appendix E.5).

Third, we have done our best to ensure fairness in the throughput comparison overall. Lafa and the AR baseline use the same backbone, GPU, and precision, and each parallel baseline uses the optimized decoding algorithm from its respective paper. We also tried FlashAttention [1], FlexAttention [3], and the standard PyTorch attention for each method, and we report the fastest kernel that the method supports.

For Fast-dLLM v2 [4], we clarify that the “block caching” that we disabled refers to the sub-block cache, not the prefix cache, which stays on in all our measurements. We turned off the sub-block cache because the original paper reports a negligible effect on throughput at small batch sizes (see Figure 6 of Fast-dLLM v2 [4]). Our measurement in Table R2 confirms this, and Fast-dLLM v2 is in fact slightly faster without the sub-block cache, so we chose this as the default. We will clearly describe these settings in the camera-ready version.

Table R2: Fast-dLLM v2 cache comparison at batch size one, averaged over GSM8K, HumanEval, and IFEval.

Sub-block cacheTPS
On77.07
Off78.68

[1] FlashAttention. https://github.com/Dao-AILab/flash-attention

[2] Hugging Face, “GPU inference,” Transformers documentation. https://huggingface.co/docs/transformers/perf_infer_gpu_one

[3] FlexAttention. https://pytorch.org/blog/flexattention

[4] Wu et al., Fast-dLLM v2: Efficient block-diffusion LLM, arXiv, 2025

Dear Reviewer,

Thank you again for the time and care that you put into reviewing our paper.

Since the discussion period ends in less than two days, we wanted to ask whether our rebuttal has resolved your concerns. If any points remain unclear, we would be glad to clarify them before the deadline.

Thanks for the response. The Fast-dLLM v2 clarification and Table R2 settle my fairness concern. On attribution, Table A helps, but the schedule's gain at 0.5B is small and the 7B ablations are still missing, so that part is only partly resolved. I'm keeping my score.

Reviewer_hKULOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Lafa, a block-autoregressive flow language model that retains next-token prediction. It introduces asynchronous denoising within blocks and supports multiple decoding methods, achieving promising results in both generation quality and inference speed.

Critical strengths
  • The paper proposes an interesting approach to flow-based language modeling. It also provides extensive experiments across different model sizes, decoding strategies, and benchmarks, with rich and encouraging results.
  • The paper includes substantial theoretical analysis and proofs to support the proposed method, providing a solid theoretical foundation for its design.
Critical weaknesses
  • The throughput comparison is not very fair. The AR baseline uses standard Hugging Face greedy decoding, while Lafa uses a more specialized implementation with fused verification and redrafting. It is unclear how much of the reported 2.54x speedup comes from the proposed method itself and how much comes from differences in implementation.
  • The benefit of asynchronous denoising is not well explained. The paper argues that denoising earlier tokens faster helps capture the left-to-right structure of language. However, the current results mainly show that tokenwise and loss-aware schedules perform better. It is still unclear whether the improvements come from the left-to-right denoising order or simply from allocating more computation to difficult noise levels.
Questions for authors
  • How much of the reported throughput improvement can be attributed to the proposed method rather than differences in the inference implementations?
  • How can we tell whether the improvements from asynchronous schedules mainly come from the left-to-right denoising order or from better allocation of computation across different noise levels?
2 comments

We thank the reviewer for the positive assessment of the method and its theoretical analysis, and for recognizing the strength of our comparisons and ablation studies. We address both concerns and the corresponding questions below.

W1 Q1.The throughput comparison is not very fair. The AR baseline uses standard Hugging Face greedy decoding, while Lafa uses a more specialized implementation with fused verification and redrafting. It is unclear how much of the reported 2.54x speedup comes from the proposed method itself and how much comes from differences in implementation. / How much of the reported throughput improvement can be attributed to the proposed method rather than differences in the inference implementations?

We appreciate the reviewer’s comment. We would like to first clarify that the AR baseline uses both KV caching and a highly optimized implementation offered by FlashAttention kernels, which is an established optimized configuration for AR inference [1] and is baked into Hugging Face as a standard [2]. We believe this optimization makes the comparison fairer to the AR baseline.

Second, we would like to emphasize that the fused verification and redrafting is a core part of the proposed method and is not merely an implementation choice that can be applied to the AR baseline. This decoding algorithm requires a model that performs both next-token prediction and block drafting simultaneously, so a plain AR model cannot use it (Section 3.3, Appendix E.5).

Third, we have done our best to ensure fairness in the throughput comparison overall. Each parallel baseline uses the optimized decoding algorithm from its respective paper, and the AR baseline and Lafa share the backbone, the GPU, and the precision. For each method, we tested FlashAttention [1], FlexAttention [3], and the standard PyTorch attention, and we used the fastest kernel that the method supports.

Lastly, regarding Q1, we point to the number of decoded tokens per forward pass (TPF). TPF counts forward passes instead of time, so it tests the decoding algorithm alone and does not depend on the kernel or the hardware. By this measure, Lafa is already 4.49× ahead of AR decoding at B=16 (Table 7). The wall-clock speedup (2.54×) is in fact smaller than this ratio, so the implementation does not inflate the gain of Lafa. We will more clearly state these inference settings in the revised paper.

W2 Q2. The benefit of asynchronous denoising is not well explained. The paper argues that denoising earlier tokens faster helps capture the left-to-right structure of language. However, the current results mainly show that tokenwise and loss-aware schedules perform better. It is still unclear whether the improvements come from the left-to-right denoising order or simply from allocating more computation to difficult noise levels. / How can we tell whether the improvements from asynchronous schedules mainly come from the left-to-right denoising order or from better allocation of computation across different noise levels?

Thank you for the question. We believe those two explanations do not compete with each other, as we detail as follows. Asynchronous denoising in Lafa uses a tokenwise learnt schedule, where for each position, the denoising loss across noise levels guides where integration steps are allocated. By "left-to-right decoding" we mean that earlier positions become less noisy sooner, giving later positions a cleaner, more informative prefix as they are denoised (Figure 3). Importantly, we do not hard-code this left-to-right schedule. We only learn the schedule of each block position from its denoising loss. Since tokens closer to the prefix are often easier to predict at a given noise level, the resulting schedules naturally and emergently follow a left-to-right order (Figure 3). This is how Lafa captures the left-to-right structure of language (Section 2.3). Therefore, the left-to-right order and the loss-aware schedule are not competing explanations for the gain. These are added novelties of Lafa specifically adapted for language data.

Our ablations in Table 2 support this view. When we share one learned schedule across all tokens (“blockwise”), the model reaches 31.9% accuracy, whereas when we learn a (left-to-right) schedule for each token (“tokenwise”), it reaches 34.5% with an equal or higher score on all tasks. The loss-aware schedule therefore gives its gain when it is learned for each position, which is where it takes the left-to-right form. In the camera-ready version we will make this mechanism and its Figure 3 visualization clearer.

[1] FlashAttention. https://github.com/Dao-AILab/flash-attention

[2] Hugging Face, “GPU inference,” Transformers documentation. https://huggingface.co/docs/transformers/perf_infer_gpu_one

[3] FlexAttention. https://pytorch.org/blog/flexattention/

Thanks for the rebuttal and it addresses most my concerns. Considering that I'm not very familiar with this area, I will keep my score at weak acceptance.

Reviewer_qerWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce Lafa, a block-autoregressive flow language model built by transforming pretrained autoregressive checkpoints into continuous-flow generators using only around 9B training tokens, demonstrated at up to 8B parameters. The design pairs a blockwise-causal formulation with asynchronous intra-block denoising driven by learned tokenwise time schedules, employs a categorical block denoiser, and adopts a joint objective (autoregressive + denoising cross-entropy + embedding squared error) enabling a single weight set to support next-token, parallel-flow, confidence-based, and self-speculative decoding, with flow-map distillation providing further compression.

Critical strengths
  1. The conversion is remarkably data-efficient: obtaining a flow LM from a pretrained AR model with roughly 9B tokens — against as much as 500B for comparable discrete-diffusion adaptations — while still scaling to 8B constitutes a substantive practical advance for flow-based language modeling.
  2. The asynchronous intra-block design is sensibly motivated by the directionality of language: learned tokenwise schedules denoise earlier positions first, injecting a left-to-right bias within an otherwise parallel block, while the blockwise-causal structure retains prefix caching.
Critical weaknesses
  1. The novelty is chiefly integrative and the contributions appear over-split, as block diffusion/autoregression, tokenwise time, flow-map distillation, self-conditioning, and confidence- or self-speculative decoding are each inherited from earlier work.
  2. Contribution 1 and Contribution 2 refer to essentially the same artifact and would be better consolidated into one more sharply delineated claim.
  3. The achievable quality declines with larger block sizes; the authors note that bigger blocks grow the denoising space exponentially and seemingly demand commensurately more training data which bounds the parallelism gains because the higher TPF at B=16 is purchased at reduced accuracy.
  4. The speed advantage is established only in a narrow serving configuration, with throughput measured on a single H100 at batch size 1 and at most 1024 generated tokens.
Questions for authors

None

4 comments

We thank the reviewer for recognizing the data efficiency of AR-to-flow adaptation and the motivation for asynchronous denoising. We address the four weaknesses below.

W1. The novelty is chiefly integrative and the contributions appear over-split, as block diffusion/autoregression, tokenwise time, flow-map distillation, self-conditioning, and confidence- or self-speculative decoding are each inherited from earlier work.

We agree that several components of Lafa have precedents, and we cite them in Section 2 and in the related work (Appendix A). Our contribution is to make these components work with a pretrained AR model at the 8B scale, while the model retains its quality and practical capabilities. These components were built for discrete formulations or for small-scale continuous models, so we modified each of them to support continuous flows at this scale, as we summarize below.

Block diffusion/autoregression. In discrete models, a block input is either a mask token or a random token from vocab, so the conversion from AR changes only the attention mask. In Lafa, a block input is Gaussian noise in the embedding space, and the backbone must receive a continuous time for each token. We therefore add adaLN with zero-initialized modulation, so training starts from the pretrained function, and we train only the direction of each embedding so that its norm cannot outgrow the noise (34.5 vs. 31.1 for frozen embeddings, Table 2).

Tokenwise time. Previous work uses prescribed tokenwise schedules [1, 2]. In our setting, the schedule has to support parallel generation within a block and also accommodate the directional dependencies that the pretrained AR model has learned. Instead of designing such a schedule by hand, we learn a separate schedule for each block position by extending the time warping of CDCD [3], so the model learns a left-to-right order on its own during training (Figure 3). The learned tokenwise schedules outperform both a learned blockwise schedule and fixed uniform schedule (34.5 vs. 31.9 in each case, Table 2).

Flow-map distillation. To our knowledge, Lafa is the first model that learns a flow map in the space of input embeddings of a language model. This step is essential for scale, because the original FMLM [4] operates in the one-hot state space, where the memory cost is prohibitive for large models. We newly prove that the block AR flow in the embedding space admits a valid flow map, and that the loss based on KL divergence still recovers it (Appendix D).

Self-speculative decoding. While self-speculative decoding is an existing technique, our contributions are (i) showing that blockwise flow training can preserve or even improve AR capabilities through a multi-token prediction effect, and (ii) using this finding to show that a continuous flow model can serve as an effective drafter. Neither of these points were observed in prior continuous methods as far as we are aware. Importantly, the learnable asynchronous time described above also raises the throughput of self-speculative decoding (2.56× vs. 2.44× with a synchronous time), as we show in Table R1.

Table R1: Self-speculative decoding acceptance and throughput for models trained with different time schedules.

Ablationtokens/forward (Acceptance)tokens/second (Throughput)Speedup
Blockwise uniform time schedule2.43 (56.11%)78.192.44×
Blockwise learned time schedule2.50 (58.67%)80.842.53×
Tokenwise learned time schedule (ours)2.54 (59.16%)81.882.56×

For self-conditioning and confidence-based decoding, we use them mostly as proposed and do not claim them as contributions. However, our work for the first time shows that priors such as self-conditioning and CFG extend to language embedding space at scale. Together, these modifications form a concrete recipe for converting pretrained AR models into flow language models at scale, which we regard as a contribution beyond the individual components.

[1] Wu et al., AR-Diffusion: Auto-regressive diffusion model for text generation, NeurIPS 2021

[2] Bansal and Sanghavi. Token time continuous diffusion for language modeling. arXiv 2026

[3] Dielemann et al., Continuous diffusion for categorical data, arXiv 2022

[4] Lee et al., Flow Map Language Models: One-step Language Modeling via Continuous Denoising, NeurIPS 2026

W2. Contribution 1 and Contribution 2 refer to essentially the same artifact and would be better consolidated into one more sharply delineated claim.

We thank the reviewer for this suggestion. We agree that the first two bullets can be merged. To more clearly represent our contributions, we will use one bullet for the merged methodology, and two for training efficiency and faster inference, respectively:

  • We introduce Lafa, a large flow language model that learns a blockwise asynchronous flow, and we identify key design choices for scalability, including architecture, training objectives, and inference schemes.
  • We scale Lafa to 7–8B parameters with 9B tokens and show that its learned flow matches or surpasses the quality of AR, that it can be distilled into flow maps, and that the training improves the next-token prediction of the pretrained AR model.
  • By jointly using the flow and AR capabilities for self-speculative decoding, Lafa attains 2.54× the throughput of a strong AR baseline at matched quality, outperforming existing methods for parallel generation. We will incorporate this into our next revision.

W3. The achievable quality declines with larger block sizes; the authors note that bigger blocks grow the denoising space exponentially and seemingly demand commensurately more training data which bounds the parallelism gains because the higher TPF at B=16 is purchased at reduced accuracy.

We agree that pure flow decoding can get harder as the block size grows (Figure 5). However, we clarify that the higher TPF at B=16 (Figure 6) is not purchased by trading off the quality of flow decoding. This TPF comes from self-speculative decoding, where the flow draft is only a proposal and each token in it is accepted only if it equals the AR prediction (Section 2.5). A weaker flow draft can therefore lower the TPF, but it cannot lower the accuracy. The reduced accuracy at B=16 that the reviewer pointed out thus comes from a slightly weaker next-token prediction (74.5% at B=16 vs. 76.3% at B=8), not from the difficulty of flow decoding. We note that Lafa nevertheless is more accurate than the AR baseline (73.9%) at every block size, while also being much faster (Figure 1).

Finally, we thank the reviewer for pointing to our sentence about an exponentially larger space and a larger training corpus. This sentence was a conjecture that we did not test, so we will remove it and state only the observed decline.

W4. The speed advantage is established only in a narrow serving configuration, with throughput measured on a single H100 at batch size 1 and at most 1024 generated tokens.

While our finding is already useful for local serving, where memory and throughput are the binding constraints, we agree that a single configuration is narrow. To extend our evaluations to serving configurations with higher concurrency and with longer generations, we additionally compared the 7B Lafa model and the AR baseline trained on the same data, to batch sizes up to 8 on the six main benchmarks, and to outputs of up to 4,096 tokens on the LongBench-Write benchmark [5], on an H100.

Table R1: Lafa and matched AR SFT throughput across batch sizes (tokens/s), averaged over six benchmarks.

Batch sizeAR TPSLafa TPSSpeedup
139.5100.22.54×
254.64143.632.63×
481.39176.812.17×
8126.41168.511.33×

Table R2: LongBench-Write throughput at longer output caps (tokens/s) at batch size one.

Output capAR TPSLafa TPSSpeedup
102438.4051.531.34×
204837.5152.301.39×
409637.2652.041.40×

As shown in Table R1, Lafa is faster than the AR baseline across all tested batch sizes, with a 2.2× speedup at batch size 4. Also, Lafa needs a smaller batch size than AR to reach the same total (system-level) throughput, as Lafa at batch size 2 generates 143.6 TPS in total, which is more than the AR baseline generates at batch size 8 (126.4 TPS). Each request in Lafa therefore runs about 4.5× faster than each request in the AR baseline (71.8 vs. 15.8 TPS). For longer generations using LongBench-Write, Table R2 shows that the speedup does not shrink as the output grows, since it is 1.34× at an output cap of 1,024 tokens and 1.40× at 4,096 tokens. (we note that the speedup is lower than in Table R1 since LongBench-Write specifically tests for open-ended writing, where accepted drafts are generally shorter than in math and code, agreeing with 1.5× on IFEval vs. 3.1× on GSM8K/HumanEval shown in Table 32).

Lastly, in line with many parallel decoders [6,7], the speedup in Table R1 becomes less prominent as the batch size grows, because sequence parallelism competes with batch parallelism for computation [8]. We believe that efficient kernels for block verification can improve this tradeoff, which we have noted as future work in the conclusion, and we regard them as an important step toward the use of Lafa beyond local serving. We will add this discussion to the revised paper.

[5] Bai et al., LongWriter: Unleashing 10,000+ word generation from long context LLMs, ICLR 2025

[6] Fu et al., Efficient-DLM: From autoregressive to diffusion language models, and beyond in speed, arXiv, 2025

[7] Wu et al., Fast-dLLM v2: Efficient block-diffusion LLM, ICLR, 2026

[8] Peng et al., How efficient are diffusion language models? A critical examination of efficiency evaluation practices, arXiv, 2025

Dear reviewer, thank you again for your time and for the detailed review.

As the discussion period is closing in less than two days, we would greatly appreciate it if we could know whether our rebuttal has properly addressed your concerns. If any concern remains, we would be happy to provide further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.