- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors study attention patterns in diffusion large language models (dLLMs) and reveal that EOS Overflow and Proximal Bias severely degrade the quality of model generations. Building on these insights, they propose a denoising scheduler optimized using Covariance Matrix Adaptation Evaluation Strategy (CMA-ES) with 393 trainable parameters. They demonstrate its competitive performance on LLaDA and Dream across several benchmarks.
Critical strengths
- Although the proposed scheduler is based on several heuristics, the analysis of attention failures in dLLMs and methodology seem solid.
- Competitive performance with other state-of-the-art methods.
- Evolutionary strategies do not require backward passes, reducing the memory footprint during optimization, making the method viable on lower-end hardware.
Critical weaknesses
- The authors do not openly disclose any weaknesses of their method, making evaluation of pros and cons and comparison with other methods more difficult.
- The evaluation methodology excludes major baselines like Unmasking Policies[1] and Dilated Unmasking Scheduler (DUS)[2], and Early Decision Matters (EDM) results are limited to few configurations, making comparison difficult.
- Nondifferentiability does not imply that CMA-ES is the best choice for their setup, the authors should justify their choice by acknowledging alternative strategies and comparing them.
[1] Learning Unmasking Policies for Diffusion Language Models, Jazbec et al., ICML 2026.
[2] Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models, Luxembourg et al., ICML 2026.
Questions for authors
- Have you tried comparing to Unmasking Policies and DUS at equal latency and forward-pass budgets?
- Have you considered training the scorer with REINFORCE/PPO over CMA-ES?
- Can you show a cross-dataset evaluation of one globally trained scheduler?