acceptodds
Withdrawn from ICLR 2027

DELA: Learning Controllable Edits for Diffusion Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion language models (DLMs) support bidirectional denoising and iterative refinement, but generation remains largely controlled by token-level masking schedules.This leaves draft-level refinement implicit and limits explicit control over when to insert missing content, delete redundant spans, revisit uncertain regions, or stop editing.We introduce Diffusion Editing Language Adapter (DELA), a plug-in framework that enables controllable draft refinement with a frozen DLM.DELA operates on clean, variable-length drafts using four explicit editing actions: Insert, Delete, Remask, and Stop.The controller determines when and where to edit the draft, while the frozen DLM fills masked regions through denoising, separating high-level edit control from token-level generation. A naive single-phase editor would be computationally expensive because arbitrary interior edits can disrupt prefix stability and limit reliable KV-cache reuse.To address this, DELA uses a two-phase decoding procedure: a fast, append-only drafting phase that supports KV-cache reuse, followed by a global refinement phase that enables targeted non-monotonic edits to the full draft. We train the controller using a staged curriculum that begins with learning local edit priors from synthetic recovery tasks and mined actions that improve draft quality. Subsequent stages refine edit selection through rollout-based outcome preferences and alignment with downstream task feedback. Experiments on mathematical reasoning and code generation show that DELA achieves gains of up to 5.5 percentage points over direct diffusion decoding while keeping the backbone frozen. DELA thus provides a modular approach to explicit draft-level control in diffusion language models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Models, Editing, LoRA

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xkLHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DELA, an external controller that refines drafts through INSERT, DELETE, REMASK, and STOP actions while keeping the diffusion backbone frozen. Training combines synthetic recovery, rollout-derived preferences, and task-specific feedback. Experiments report improvements on mathematical reasoning and code generation using append-only drafting followed by sparse global refinement.

Critical strengths
  1. The controller separates structural editing from token generation without updating the backbone. Reported LLaDA gains span four tasks, including HumanEval improvement from 37.2% to 42.7%, with latency increasing from 5.43 to 5.74 seconds.
  2. The shared-draft baseline, REMASK-only comparison, and training-stage analysis help explain the gains. The results indicate selective local repair, with remasking accounting for most improvements, rather than broadly successful structural rewriting. The paper also acknowledges that extra inference compute is not fully isolated.
Critical weaknesses

The incremental benefit of structural editing remains insufficiently established.

The contribution of additional inference compute is not fully isolated.

The task-specific training data are insufficiently specified to establish held-out improvement.

Questions for authors

Please identify the Stage 3 training and validation sets for each backbone/task, document overlap checks against benchmark prompts, solutions, and tests, and explain how checkpoints and inference settings were selected.

Reviewer_fPuTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors introduce DELA, an adapter on top of masked diffusion language models (DLM) that controls iterative refinement by identifying tokens to remask or delete, inserting spans of masked tokens, or deciding to stop editing. Importantly, DELA is orders of magnitude smaller than the masked DLM it augments, and uses the DLM to unmask tokens. The authors consider a sophisticated 3-phase training strategy to train the adapter. First, DELA learns to undo synthetic corruptions. Then, it trains a separate selector matched to the underlying DLM that scores which edit actions are useful. Finally, RL training is applied with task-specific rewards. The authors instantiate DELA with a draft-first then-refine inference strategy and report accuracy improvements on math and coding at a modest latency increase compared to decoding without refinement.

Critical strengths

The paper tackles an important missing piece in standard diffusion language models: how to enable iterative refinement given the fixed-size canvas on which DLMs typically operate. Instantiating a low-cost policy to select locations to insert/delete/remask is an elegant approach to reuse provided DLMs that take substantial computational resources to train.

Critical weaknesses

The paper is very hard to follow. A substantial piece of the architecture is the selector, which is barely treated until the learning pipeline is presented. Also, the description of stage 1 and stage 2 is quite messy. At the end of stage 1 "mined" actions are described, but not to a sufficient degree to actually understand how they are mined and what exactly differentiates them from the standard synthetic actions. The paper is extremely vague in this section, e.g., mentioning "target-aware rollout evaluation" but on "task-agnostic general text states". It is unclear what much of this means. In stage 2, the authors describe "task-agnostic outcome scores" as "structural recovery and denoising-consistency metrics", again without any additional explanation, or mention "short-horizon outcomes" without any description.

Questions for authors

Questions are ordered according to relevance.

  1. Can you completely rewrite Section 3.2 to make the 3 stages clear? Between each stage, identify precisely what you are doing and how it differs from the previous stages. Try to avoid vague/meaningless words, but actually describe your precise method.
  2. Can you early on introduce the architectural choice of splitting the policy into an action proposer module and an action selector module? Explain why this choice was adopted. Also highlight consistently in Section 3.2 where you train which modules.
  3. There seems to be a discrepancy between the parameterization of INSERT/DELETE (via position + length) and REMASK (via logits). If DELA anyhow has to produce logits at all token positions, why don't you just predict the action token-wise and in case INSERT is selected additionally have a length head? DELETE could be handled token-wise just like the remasking, and STOP could follow naturally if no action passes the activating threshold. Can the authors ablate on their choice of parameterization?
  4. In lines 400-401, you describe an ORACLE selection baseline. How can ORACLE select the best (step-wise?) candidate if the task evaluator only scores final results after multi-step editing?
  5. For the synthetic training in stage 1 with REMASK, you do not specify how you corrupt the sequence xx to x~\tilde x. Is this just a step of uniform diffusion?
  6. Concerning Section 4.3, can the authors repeat the experiment with Fast-dLLM for initial drafting followed by their method for refinement? Shouldn't this provide a much better latency-accuracy tradeoff?
  7. In Equation (1), can application of Dτ\mathcal D_\tau involve multiple calls to the DLM since unmasking is threshold-based?
  8. On line 265 and 352 you write "Eq. equation ...", which is a formatting mistake. I suspect this formatting error may occur also in other places.
Reviewer_saWNOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DELA, an external edit controller that refines drafts produced by a frozen diffusion language model. The controller chooses among INSERT, DELETE, REMASK, and STOP, and the frozen backbone fills the masked regions. Decoding has two phases: an append-only drafting phase that reuses the KV cache, and a global refinement phase that edits the full draft. The editor and selector (about 200M parameters in total) are trained in three stages: synthetic recovery examples and mined edits, rollout-based preferences on general text, and task feedback. With a frozen LLaDA-1.5, DELA improves GSM8K, MATH, HumanEval, and MBPP by up to 5.5 points over direct decoding, and a separately trained controller gives smaller gains on Dream-7B.

Critical strengths

Separating edit decisions from token generation is a clean idea. It turns a frozen diffusion model into an editable generation system and improves results on four math and code benchmarks without updating the backbone. The three-stage training procedure is intuitive and easy to follow.

Critical weaknesses
  1. Limited novelty. The training procedure combines familiar pieces: imitation of synthetic and mined edits, rollout-based policy improvement, and task-reward alignment. The contribution mainly lies in integrating them into a diffusion editing framework. The paper also does not compare this preference-distillation procedure with standard RL training of the same edit policy, so it is unclear whether the proposed procedure offers an advantage.

  2. Missing cost-controlled comparisons. DELA adds a 200M controller, three training stages, and extra inference latency over direct decoding. The paper does not show whether fine-tuning the backbone, or spending comparable extra compute on decoding, would give a better quality-cost trade-off. The ablations also do not establish that the three stages and the controller size are necessary.

Other concerns: the Dream-7B result uses a separately trained controller, so the plug-in design is not yet shown to transfer across backbones. The editor and selector architectures are not specified well enough to reproduce, and the 200M size is not justified. The relation between fast drafting, local remasking, global refinement, and caching is hard to follow.

Overall, the limited novelty and the missing cost-controlled comparisons leave the contribution insufficiently supported, and I lean toward rejection in its current form.

Questions for authors
  1. How does the staged preference-distillation procedure compare with standard RL training of the same edit policy under the same budget?
  2. How does DELA compare with fine-tuning the backbone, or with spending the same extra compute on decoding, at a matched training and inference budget?
  3. Can a controller trained on LLaDA-1.5 be used on Dream-7B without retraining? If not, how much data and compute does retraining take?
  4. Why should a 200M controller be able to judge where an 8B backbone's draft is wrong? What are the editor and selector architectures, and how does performance change with controller size?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.