acceptodds
Withdrawn from ICLR 2027

Optimizing Denoising Trajectories in dLLMs: A Lightweight Evolutionary Heuristic Approach

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to conventional Auto-Regressive (AR) Large Language Models (LLMs). By leveraging bidirectional attention and parallel decoding, dLLMs enable more efficient generation. However, they require a carefully designed denoising scheduler at inference time (absent during training) whose choice significantly impacts generation quality. While confidence-based heuristic schedulers have shown strong empirical performance, they suffer from two critical failure modes: EOS Overflow and Proximal Bias. Through in-depth analysis of the Transformer's attention patterns, we reveal that these failures stem from certain positions assigning disproportionately high attention weights to invalid tokens (e.g., [MASK] and [EOS]), which produce misleading confidence signals. Building on this insight, empirical evidence shows that valid attention scores can provide complementary guidance to conventional confidence-based heuristics, yet no single metric consistently excels across all scenarios, implying that the optimal denoising trajectory is highly context-dependent. To address this problem, we propose a lightweight evolutionary heuristic scheduler optimized using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Our scheduler dynamically integrates multiple heuristic features with a contextual mean-field embedding, while requiring only 393 trainable parameters. Evaluated on LLaDA and Dream across four reasoning and planning benchmarks, our method consistently outperforms strong baselines, including conventional heuristics, block auto-regressive methods, and recent State-Of-The-Art (SOTA) approaches. To the best of our knowledge, it represents the most parameter-efficient neural scheduler to date. Our code is available at anonymous repository https://anonymous.4open.science/r/Evo-Denoiser-CE0D .

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Model, Denoising Scheduler

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HraDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors study attention patterns in diffusion large language models (dLLMs) and reveal that EOS Overflow and Proximal Bias severely degrade the quality of model generations. Building on these insights, they propose a denoising scheduler optimized using Covariance Matrix Adaptation Evaluation Strategy (CMA-ES) with 393 trainable parameters. They demonstrate its competitive performance on LLaDA and Dream across several benchmarks.

Critical strengths
  • Although the proposed scheduler is based on several heuristics, the analysis of attention failures in dLLMs and methodology seem solid.
  • Competitive performance with other state-of-the-art methods.
  • Evolutionary strategies do not require backward passes, reducing the memory footprint during optimization, making the method viable on lower-end hardware.
Critical weaknesses
  1. The authors do not openly disclose any weaknesses of their method, making evaluation of pros and cons and comparison with other methods more difficult.
  2. The evaluation methodology excludes major baselines like Unmasking Policies[1] and Dilated Unmasking Scheduler (DUS)[2], and Early Decision Matters (EDM) results are limited to few configurations, making comparison difficult.
  3. Nondifferentiability does not imply that CMA-ES is the best choice for their setup, the authors should justify their choice by acknowledging alternative strategies and comparing them.

[1] Learning Unmasking Policies for Diffusion Language Models, Jazbec et al., ICML 2026.

[2] Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models, Luxembourg et al., ICML 2026.

Questions for authors
  1. Have you tried comparing to Unmasking Policies and DUS at equal latency and forward-pass budgets?
  2. Have you considered training the scorer with REINFORCE/PPO over CMA-ES?
  3. Can you show a cross-dataset evaluation of one globally trained scheduler?
Reviewer_Ue3gOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates EOS overflow and proximity bias in diffusion language model decoding and proposes a 393-parameter scheduler that combines confidence, attention, and positional features. The scheduler is optimized with CMA-ES using final-answer correctness while keeping the backbone frozen, and is evaluated on two models across four reasoning and planning benchmarks.

Critical strengths
  1. Combining complementary heuristic signals in a compact scorer is a practical idea, and optimization requires neither backbone fine-tuning nor gradients through the decoding process.
  2. Experiments cover two dLLM backbones and four tasks, with reported improvements in many settings with limited decoding steps.
Critical weaknesses
  1. The mean-field branch appears mathematically redundant for the stated ranking operation. Expanding Eq. (11) gives z_{t,i} = u^T y_{t,i} + v^T ybar_t + b, where ybar_t is shared by all masked positions. Its contribution is therefore a common additive offset that cancels in pairwise score differences and cannot affect the Top-k selection in Eq. (2), even after adding the affine bypass. The first-layer nonlinearities occur before the pooled context is injected and do not resolve this cancellation, contrary to the argument in Appendix D, P2. This undermines the claimed role of the explicit mean-field context, although the input features themselves can still carry contextual information. Please clarify whether the implementation differs from the equations, correct the argument, and perform a fixed-weight ablation that zeros only the mean-field contribution; retraining an ablated architecture does not isolate its effect on ranking.
  2. Table 1 places the imported EDM scores of 56.8 on GSM8K, 22.8 on MATH, and 43.8 on Countdown under L=128, T=32, whereas the original paper reports them at L=256, T=32. Please correct this configuration mismatch and reassess the comparisons under matched settings.
  3. Parameter count alone does not establish practical efficiency: CMA-ES requires many backbone rollouts, and extracting attention features may add inference overhead. Please report total training GPU-hours and end-to-end inference latency or throughput against the main baselines.
  4. The attention plots and correlations do not establish that excessive attention to invalid tokens causes EOS overflow and proximity bias. Please add controlled interventions and aggregate trajectory statistics across examples, or narrow the causal claims.
  5. The ablations do not establish whether the nonlinear scorer is necessary beyond a learned linear combination, or which feature groups drive the gains. Please include a linear-only scorer, targeted attention/position feature ablations, and variability across independent CMA-ES runs.
Questions for authors

See above.

Reviewer_WF8DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose a lightweight scheduler approach for optimizing trajectories in diffusion large language models such as LLaDA.

These models decode an answer by iteratively filling in masked positing and a scheduler decides with masks to fill in at each step. Traditional schedules select the positions where the model is most confident. Because of that, they fill the end of the answer too early and keep filling positions that are neighbors to the ones they filled in the previous step. The paper shows that different simple schedulers solve successfully different questions.

The authors propose a very lightweight scheduler (only 393 parameters) that combines several simple signals per position, such as confidence, margin, attention from different layers, position features with a summary of all masked positions. The scheduler is trained using Covariance Matrix Adaptation Evolution Strategy (CMA-ES), which is a gradient-free search that used only whether the final answer is correct.

The experiments were performed using two different diffusion models (LLaDA-8B and Dream-7B) and the papers shows results on four math and logic reasoning and planning benchmarks. The results show that the proposed scheduler yields better performance (in most settings) than all reported baselines including confidence-based heuristics, block-wise decoding and recent methods

On GSM8K, MATH-500, Countdown and StrategyQA with few decoding steps, it beats confidence-based heuristics, block-wise decoding and recent state-of-the-art approaches (EDM, CCD, AGDO and Suffix Anchor) in most settings, on both LLaDA-8B and Dream-7B.

Critical strengths

[S1] The paper shows that the proposed scheduler has clear gains compared to existing schedulers. Tables 1 and 3 show that in the majority of the settings across two models the proposed learned scheduler performs best.

[S2] Simple idea that seems easy to use and extremely lightweight in terms of learned parameters.

[S3] Overall, I found that the paper was well written and easy to follow.

Critical weaknesses

[W1] The proposed scheduler is tuned on the training data of each benchmark, while almost every baseline (except EDM) is used without any tuning or use of in-domain training data. Therefore, it's not clear if the gains come from the extra tuning or the better design. I would expect some simple baselines for comparison, such as a simple weighted sum of the same features, or selecting the best existing heuristic for each setting on the training data.

[W2] I agree that the schedule is lightweight in terms of the number of parameters but the training cost of the scheduler is not reported in the paper. Each training requires thousands of answers from an 8B diffusion LLM and the paper does not report GPU-hours and comparison with the cost of other learned schedulers.

[W3] It is not clear if the attention, which is the main signal of the approach, is useful for the scheduler. The paper does not ablate on that by testing the scheduler without the attention features. Intuitively, the position features may be enough for learning to avoid early end tokens and alone can explain much of the gain.

[W4] A scheduler trained on Countdown performs worse than the simple Prob Margin on MATH-500. This means that the scheduler may need to be retrained for each benchmark and task.

[W5] The training of the scheduler may be noisy and unstable since it's done with only 10 rounds of search over 393 parameters and each candidate scored on 100 questions. Because of that, I would expect to see the performance of several runs and reporting of the training variance. Some of the reported gains are negligible on GSM8K and it's not clear if they are within noise.

Questions for authors

[Q1] Could you ablate on the input signals and report the results of the scheduler without the attention features and without the positions features? [Q2] Can you include important and simple baselines that are trained on the same data? (e.g., weighted sum of the same features, choose the best existing heuristic per setting) [Q3] Can you report the mean and variance over several training runs for a few settings? [Q4] How does the accuracy changes from the beginning over the several search rounds? [Q5] How many GPU-hours does each training take?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.