- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Clock Diffusion, a semi-autoregressive framework for continuous diffusion language models based on position-dependent noise schedules. The authors further introduce ClockDLMs and Cache Grab samplers to improve generation quality and inference efficiency. Experiments across multiple benchmarks demonstrate the effectiveness of the proposed framework.
Critical strengths
The paper presents a comprehensive framework for continuous diffusion language models, covering model formulation, efficient training, and accelerated inference. The experimental evaluation is also extensive. The strong results across different benchmarks demonstrate the effectiveness of the proposed framework.
Critical weaknesses
- Limited evaluation at larger scales. The experiments focus on relatively small models. It remains unclear whether the proposed framework can maintain its advantages when scaled to larger language models.
- Limited practical efficiency evaluation. The paper mainly evaluates inference efficiency using NFE, without sufficient wall-clock latency and throughput measurements. These results would help establish whether the reported computational savings translate into practical speedups.
Questions for authors
It would be valuable to provide a more comprehensive evaluation of practical inference efficiency, including wall-clock latency and throughput. In particular, a comparison with recent discrete or hybrid diffusion approaches such as TiDAR would help clarify the performance and efficiency trade-offs between continuous and discrete diffusion language models. It is okay if continuous diffusion does not outperform these methods, as it is still an emerging research direction. If computational resources are limited, it is also acceptable not to conduct larger scale experiments.
4 comments
Thank you for the detailed response and additional experiments,and I have raised my score accordingly.
Follow up: TiDAR experiment results
As promised in our response to Q1, here are the results of the controlled comparison with TiDAR (Liu et al., 2025).
| Model | GSM8K acc. (%) () | Tokens/forward () | Tokens/s () | ms/forward () | |
|---|---|---|---|---|---|
| AR baseline | – | 64.3 | 0.99 | 158 | 6.3 |
| TiDAR | 4 | 61.6 | 3.56 | 509 | 7.0 |
| TiDAR | 8 | 62.0 | 5.84 | 805 | 7.3 |
| TiDAR | 16 | 60.0 | 7.55 | 1009 | 7.5 |
| TiDAR | 32 | 60.0 | 7.05 | 886 | 8.0 |
| ClockDLM + Cache Grab-SSD | 16 | 65.9 | 4.05 | 347 | 11.7 |
| ClockDLM + Cache Grab-SSD | 32 | 67.6 | 4.39 | 371 | 11.8 |
| ClockDLM + Cache Grab-SSD | 64 | 64.8 | 3.67 | 302 | 12.2 |
- Accuracy: Every ClockDLM + Cache Grab-SSD configuration (64.8–67.6%) is more accurate than every TiDAR model (60.0–62.0%) and than the AR baseline, whereas TiDAR falls 2-4 points below the AR model it was initialized from.
- Throughput: TiDAR is currently 1.4-2.7× faster in tokens per second than our fastest Cache Grab-SSD configuration (), for two reasons. First, its forward passes are cheaper (7.0-8.0 ms vs. 11.7-12.2 ms), consistent with the per-step cost of our current implementation discussed in W2. Second, at , TiDAR produces more tokens per forward (5.8-7.6 vs. 3.7-4.4) because it drafts and verifies in a single forward pass, whereas Cache Grab-SSD uses separate passes. At matched , these two factors contribute 1.5× and 1.6×, respectively (2.4× overall).
Thank you
Thank you for your continued engagement with our work and for raising your score.
Sincerely,
The Authors
Response to Reviewer qUKk
We thank the reviewer for their positive comments and for their constructive feedback. Below we address the specific issues raised. Please note that all paper revisions discussed below have been incorporated into an updated PDF posted here and are marked in
bluefont.W1: Limited evaluation at larger scales
Thank you for this suggestion. To evaluate our method at larger scales, we used the released 0.5B and 7B checkpoints of Lafa, from the recent work “Large Flow Language Models” (Chavan et al., 2026). Lafa models are block semi-autoregressive (SAR) continuous flow language models, adapted from Qwen2.5-Instruct, with a block size of 4. From each released checkpoint, we ran two fine-tuning (FT) runs that differ only in the clock and training mask:
Sliding-window FT matches or outperforms block FT on all six model/benchmark pairs, with gains of up to 9.1 points (7B, IFEval), indicating that the advantage of our sliding-window clocks persists at the 7B scale and carries over to a continuous language model beyond our ClockDLMs.
0.5B Lafa Model
7B Lafa Model
Experimental setup notes:
W2: Report additional throughput metrics other than average NFEs/sequence
This suggestion is well taken. In the revised appendix, we added versions of Figures 4, 5, and 6 that use tokens per forward (↑) and tokens per second (↑) on the horizontal axes, in place of the NFEs/seq (↓) used in the main-text figures. These are Figures 9, 10, and 11 in the new Appendix G.4 (“Additional Throughput Metrics”), which also defines both metrics and the measurement setup (batch size 1, single H100, bfloat16).
Against the diffusion baselines, the trends of the original figures carry over to both metrics.
For example, at S=32 and N=4, ClockDLM reaches 53.8% accuracy with 3.9× fewer NFEs than BD3LM at comparable accuracy (52.9%; Figure 4), which translates into 2.6× higher tokens per second (529 vs. 202; Figure 9).
We note that the wall-clock gains are smaller than the NFE savings, however, because each ClockDLM decoding step (model forward plus sampler update) currently takes about 12 ms, vs. about 6 ms for AR and 8–9 ms for BD3LM. As a result, at S=32 and N=8, for example, ClockDLM uses 3.4× fewer NFEs than AR but is only 1.7× faster in tokens per second (270 vs. 158), at 3.8 points lower accuracy (60.5% vs. 64.3%). We expect optimized kernels and inference serving to reduce the per-step overhead and widen these gains.
The Cache Grab samplers translate their NFE savings into wall-clock speedups over standard ClockDLM decoding (Figure 10). Cache Grab-Confidence (η=0.7) increases tokens per second by 2.9–4.2×, with accuracy changes between −2.4 and +0.4 points. Cache Grab-SSD, with the jointly trained models of Appendix G.3, increases tokens per second by 4.6–5.4× while also improving accuracy, and every Cache Grab-SSD configuration exceeds AR in both accuracy and tokens per second (1.9-2.3×).
Q1: How does ClockDLM compare to works, such as TiDAR in terms of efficiency-performance trade-offs?
Thank you for suggesting this comparison. For a controlled comparison with TiDAR (Liu et al., 2025), we are currently conducting the following experiment:
We will post the results in this thread before the end of the discussion period as soon as they are available. Please let us know if this intended experiment matches your expectations.