- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Lafa converts pretrained autoregressive language models into block-autoregressive flow models while retaining next-token prediction. It combines bidirectional attention within noisy blocks, learned tokenwise time schedules, and joint autoregressive and denoising objectives. The resulting model supports iterative flow decoding, distilled flow-map decoding, confidence decoding, and self-speculative decoding.
Experiments cover models from 0.5B to 8B parameters, adapted using approximately 9B tokens. The strongest speed result comes from one-step block drafting with fused self-verification: Lafa reports 2.54× the decoding throughput of the matched AR baseline at comparable average quality on three tasks.
Critical strengths
-
The paper provides a concrete recipe for scaling continuous flow language models. Converting pretrained AR checkpoints to 7–8B flow models while retaining next-token prediction is a useful result. The blockwise formulation also preserves prefix caching, making the approach practical to implement.
-
The self-speculative results show a meaningful speed–quality tradeoff. Table 32 reports actual decoding throughput rather than relying only on tokens per forward. At B=8, Lafa reaches 93.8 tokens/s with an average score of 76.3, compared with 39.5 tokens/s and 73.9 for matched AR SFT. Comparisons with MTP, MARS, and diffusion models strengthen the evidence beyond the AR baseline.
Critical weaknesses
-
The ablations do not establish which flow-specific designs drive the main acceleration result. Table 2 evaluates training choices at 0.5B using 32-step flow decoding, whereas the headline speedup uses a 7B model with one-step drafting and self-verification. Improvements in iterative flow quality do not directly establish improvements in draft acceptance or decoding speed. This leaves the contribution of asynchronous schedules and the flow-training recipe to the strongest practical result unresolved.
-
The throughput comparison leaves questions about implementation fairness. Appendix E.5 uses standard cached Hugging Face generation for AR and a specialized fused verification-and-redrafting implementation for Lafa. It also disables block caching for Fast-dLLM v2 without clearly explaining this choice. These settings do not invalidate the measurements, but they make it difficult to separate the method’s advantage from differences in inference optimization. Since speed is central to the paper, this materially affects my assessment.
Questions for authors
-
Contribution clarification: Do existing experiments evaluate synchronous schedules or a simpler block-prediction objective under the same fused self-speculative decoder? Acceptance rates and wall-clock results at 7B would help establish which flow-specific choices contribute to the reported speedup.
-
Evaluation clarification: Why was block caching disabled for Fast-dLLM v2, and what inference optimizations were used for the AR and parallel-generation baselines? Please clarify whether the reported advantage holds under comparable optimization settings.
4 comments
W2 Q2. The throughput comparison leaves questions about implementation fairness. Appendix E.5 uses standard cached Hugging Face generation for AR and a specialized fused verification-and-redrafting implementation for Lafa [...] Since speed is central to the paper, this materially affects my assessment. / Why was block caching disabled for Fast-dLLM v2, and what inference optimizations were used for the AR and parallel-generation baselines? Please clarify whether the reported advantage holds under comparable optimization settings.
We appreciate the reviewer’s comment. We would like to first clarify that the AR baseline uses both KV caching and a highly optimized implementation offered by FlashAttention kernels, which is an established optimized configuration for AR inference [1] and is baked into Hugging Face as a standard [2]. We believe this optimization makes the comparison fairer to the AR baseline.
Second, we would like to emphasize that the fused verification and redrafting is a core part of the proposed method and is not merely an implementation choice that can be applied to the AR baseline. This decoding algorithm requires a model that performs both next-token prediction and block drafting simultaneously, so a plain AR model cannot use it (Section 3.3, Appendix E.5).
Third, we have done our best to ensure fairness in the throughput comparison overall. Lafa and the AR baseline use the same backbone, GPU, and precision, and each parallel baseline uses the optimized decoding algorithm from its respective paper. We also tried FlashAttention [1], FlexAttention [3], and the standard PyTorch attention for each method, and we report the fastest kernel that the method supports.
For Fast-dLLM v2 [4], we clarify that the “block caching” that we disabled refers to the sub-block cache, not the prefix cache, which stays on in all our measurements. We turned off the sub-block cache because the original paper reports a negligible effect on throughput at small batch sizes (see Figure 6 of Fast-dLLM v2 [4]). Our measurement in Table R2 confirms this, and Fast-dLLM v2 is in fact slightly faster without the sub-block cache, so we chose this as the default. We will clearly describe these settings in the camera-ready version.
Table R2: Fast-dLLM v2 cache comparison at batch size one, averaged over GSM8K, HumanEval, and IFEval.
| Sub-block cache | TPS |
|---|---|
| On | 77.07 |
| Off | 78.68 |
[1] FlashAttention. https://github.com/Dao-AILab/flash-attention
[2] Hugging Face, “GPU inference,” Transformers documentation. https://huggingface.co/docs/transformers/perf_infer_gpu_one
[3] FlexAttention. https://pytorch.org/blog/flexattention
[4] Wu et al., Fast-dLLM v2: Efficient block-diffusion LLM, arXiv, 2025
Dear Reviewer,
Thank you again for the time and care that you put into reviewing our paper.
Since the discussion period ends in less than two days, we wanted to ask whether our rebuttal has resolved your concerns. If any points remain unclear, we would be glad to clarify them before the deadline.
Thanks for the response. The Fast-dLLM v2 clarification and Table R2 settle my fairness concern. On attribution, Table A helps, but the schedule's gain at 0.5B is small and the 7B ablations are still missing, so that part is only partly resolved. I'm keeping my score.
We thank the reviewer for the positive assessment, particularly for recognizing the significance of scaling continuous flow language models and self-speculative speed–quality results. We address the two questions below.
W1 Q1. The ablations do not establish which flow-specific designs drive the main acceleration result [...] This leaves the contribution of asynchronous schedules and the flow-training recipe to the strongest practical result unresolved. / Do existing experiments evaluate synchronous schedules or a simpler block-prediction objective under the same fused self-speculative decoder? Acceptance rates and wall-clock results at 7B would help establish which flow-specific choices contribute to the reported speedup.
We first clarify that a direct comparison between training variants is less straightforward in self-speculative decoding than in flow decoding. In flow decoding, we can compare the variants in terms of accuracy at a fixed NFE. Yet, in self-speculative decoding, it is difficult to fix either speed or accuracy for a fair comparison. The accuracy is the same with the next-token prediction performance, while throughput depends on the acceptance rate of speculative tokens. Both come from the same weights, so a single training choice changes both the flow (drafter) and the next-token prediction (verifier). Each ablation checkpoint therefore differs in accuracy and in acceptance at the same time, and different training choices lead to different trade-offs between speed and accuracy. Nevertheless, in Table A we show the results of the 0.5B ablations of Table 2 under self-speculative decoding, including the synchronous (blockwise) schedules that the reviewer requested. We could not retrain the 7B model with these ablations within the rebuttal period, and we will try our best to add these results to the camera-ready version.
Table A: 0.5B self-speculative decoding ablation. Speedup is relative to AR decoding, and accuracy is averaged over GSM8K, HumanEval, and IFEval.
The last three rows show a clear example of the difficulty described above. Since they are trained without the AR loss, their accuracy collapses to about 5%, yet they accept about 94% of the drafted tokens and reach the highest speedup.
Among the variants with the AR loss, the tokenwise learned schedule gives the highest acceptance and throughput among the time schedules (2.43, 2.50, and 2.54 tokens/forward), so the asynchronous schedule contributes to the speedup over the synchronous schedule. For the embeddings, the three variants are close in throughput (2.55× to 2.60×), and the unconstrained variant is slightly ahead at 0.5B. Our final recipe thus remains close to the best in throughput while preserving accuracy in this setting, and it gives clear gains in flow mode (Table 2). We therefore believe that our design choices are reasonable for a single model that performs well under all decoding modes.
For a comparison on a simpler block-prediction objective, Table 7 in the appendix already provides the closest comparison at 7B. MARS is trained on the same data to predict a block from masked tokens, and we count its forward passes in the same way as for fused decoding in Lafa. At B=8, Lafa commits 4.05 tokens/forward, whereas MARS commits 2.99 at comparable accuracy.