- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents a training-free decoding framework called CoCom for masked diffusion language models. The main idea is to jointly determine which predicted tokens should be permanently committed and how strongly the remaining predictions should be used as revisable context for subsequent denoising steps. The authors further derive a computable upper bound on total correlation using only the model's marginal distributions. Based on this bound, CoCom selects the largest feasible commitment prefix and the corresponding retention probability. The resulting retention probability is then used as the interpolation strength for updating the pending token embeddings.
Critical strengths
Pros
- The mathematical formulation and the proposed method are coherent. I found the mathematical derivations clear, and the algorithmic design is well motivated by the theoretical analysis.
- The experimental results are promising. CoCom outperforms many of the compared accelerated decoding methods across different models and tasks. The reduction in mean NFE is substantial, and the reported wall-clock speedups over Native decoding are also encouraging.
- The ablation studies are extensive and useful.
Critical weaknesses
Cons
- The experimental setup for the Native baselines is not entirely clear. CoCom uses block-wise diffusion with a block length of 64. However, the authors do not clarify whether the Native baselines use the same block-wise setting or full-sequence diffusion. Since the block length can significantly affect decoding performance, this difference in configuration could make the comparison less fair.
Questions for authors
- My only question is related to the weakness. Could the authors clarify the specific decoding setup used for the Native DLM baselines in the Main Results Table? As mentioned in Table 6, CoCom uses block-wise decoding with a block length of 64, but the block length for Native is not specified. I also noticed that the reported GSM8K accuracy for LLaDA-1.5-8B is substantially lower than the officially reported results under block-wise decoding. Does the Native baseline use the same block-wise setting as CoCom, or full-sequence diffusion?
3 comments
Thank you for the clarification and the additional experiments. These are good results showing CoCom gets comparable quality results to native baseline at the same time produces significant speedup. I have increased my score to clear accept.
We sincerely appreciate your positive feedback and updated assessment, and are glad that the additional experiments addressed your concerns. Thank you again for your constructive review, and we will make sure to polish the manuscript accordingly.
We sincerely thank the reviewer for the positive assessment and for identifying an important omission in our experimental description. We clarify the Native settings below and provide an additional comparison with matched block lengths.
Native used full-sequence diffusion over a 256-token generation canvas, with 256 forward evaluations. CoCom used block length 64. We agree that this difference should have been explicit, since block length can affect accuracy independently of the acceleration method.
Our evaluation fixes the generation length at 256 tokens across backbones and methods to provide a common output budget. Full-sequence diffusion is consistent with the sampling regime in the original LLaDA-Instruct main results [1], as well as the Dream and DiffuCoder models [2, 3]. For LLaDA-1.5 specifically, the official evaluation searches sampling strategies, generation lengths, and block lengths, and reports the best results [4, Appendix D.4].
To address your concern about the fairness of this comparison, we have additionally evaluated Native with the same block length of 64 as CoCom on all four LLaDA-1.5 benchmarks, keeping its original prompts, generation length, prediction rule, and fixed 256-forward budget. The results are shown below. In the revised manuscript, we specify Native's block length in the decoder-protocol table, add this matched-block comparison to the appendix, and summarize its findings in the main results.
Native block64 reaches 82.03% on GSM8K, close to the officially reported 83.3%. With matched block lengths, CoCom achieves higher accuracy on three tasks. On GSM8K, its accuracy is 1.06 percentage points lower with 76.0% fewer forward evaluations (61.39 vs. 256). These results support CoCom's favorable accuracy-efficiency tradeoff.
References
[1] Nie, S., et al. Large Language Diffusion Models. NeurIPS, 2025.
[2] Ye, J., et al. Dream 7B: Diffusion Large Language Models. arXiv preprint arXiv:2508.15487, 2025.
[3] Gong, S., et al. DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation. ICLR, 2026.
[4] Zhu, F., et al. LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models. arXiv preprint arXiv:2505.19223v2, 2025.