acceptodds
Under review as a conference paper at ICLR 2027

Draft-Guided Constrained Diffusion for Structured Generation in LLMs

Abstract

Structured generation requires language models to produce outputs that are both task-correct and compliant with hard structural constraints, such as JSON schemas, often within a fixed output budget. Constrained decoding (CD) enforces such constraints in autoregressive (AR) models, but restricting the next-token distribution can alter the model's solution and may still leave the output incomplete when the generation budget is exhausted. We observe that template-based diffusion offers a complementary mechanism, however, using a diffusion language model (dLLM) directly still limits task accuracy to the dLLM's own problem-solving ability. We propose Draft-Guided Constrained Diffusion (DGCD), which separates these roles. An AR model first produces an unconstrained draft, and a dLLM then expresses its content within a fixed schema template. For feasible template-expressible schemas under our validity assumptions, DGCD guarantees structural validity by construction, while its task accuracy depends on faithfully preserving the draft's answer. Across six AR drafters from 1B to 14B on mathematical reasoning and structured extraction, DGCD achieves observed structure adherence on every evaluated setting and improves average accuracy over draft-conditioned constrained decoding by up to percentage points. DGCD also remains robust under tight output budgets and is faster than DCCD in our latency evaluation.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
diffusion llms, constrained decoding, structured generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nukgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose DGCD for schema-constrained generation: an AR model writes an unconstrained draft, and a masked diffusion LM then fills the content slots of a template whose structural tokens are fixed in advance, conditioned on the input and the draft. On GSM8K, MATH500, and two extraction benchmarks with six AR drafters (1B–14B), they report 100% schema adherence and higher accuracy than constrained decoding and DCCD, robustness to tight token budgets, and lower latency.

Critical strengths
  • The method target a known problem with a well-motivated connection. The paper’s insight is that solving and formatting need not share a generation mechanism: the AR model solves freely, and schema infilling is offloaded to a dLLM that conditions on structure fixed in advance.
  • Results cover six AR drafters from two model families (1B–14B) on four benchmarks spanning reasoning and extraction. The direction of the effect is consistent across this grid
Critical weaknesses
  • The accuracy gain is not isolated to the proposed mechanism. DGCD differs from CD by adding a draft pass, and from DCCD by changing the formatter model, fixing the template, and using diffusion, all at once. The paper attributes the gain to diffusion, but the needed controls are absent: -- A draft followed by unconstrained reformatting with the same AR model, -- Draft-conditioned AR infilling of the same fixed template with the same slot caps
  • In Table 2, DCCD reuses the drafter as its formatter, while DGCD always adds LLaDA-8B. The “1B” DGCD row is a 9B system compared with a 1B one. The structurer also sees the original input, so on extraction it may be solving the task itself.
  • Accuracy is reported with LLaDA-8B, but latency and parameter efficiency use a fine-tuned 1.5B structurer whose accuracy is not reported.
  • Appendix H formalizes the distortion of AR constrained decoding but gives no corresponding analysis for DGCD, which applies the same mask-and-renormalize step at content positions (Eq. 3) and additionally samples multiple positions per step from per-position marginals. The paper does not state what distribution DGCD samples from, nor give conditions under which the draft’s answer is preserved.
Questions for authors
  • What accuracy does the same AR model reach if it drafts and then reformats into JSON with no grammar constraint?
  • What accuracy does draft-conditioned AR infilling reach on the same fixed template with the same slot caps?
  • What is DGCD’s latency with LLaDA-8B, the structurer used for the accuracy results?
  • What distribution does DGCD’s stage 2 sample from, and how does its deviation from the draft-conditioned, validity-conditioned dLLM distribution compare with Eq. 8 for CD?
5 comments
AuthorsOpenReview ↗

Response to Reviewer nukg [Part I]

Response to Reviewer nukg — Score 2/4

We thank the reviewer for identifying the controls needed to clarify the contributions of draft conditioning, template allocation, and the choice of structurer. Below, we report the additional AR-reformatting and LLaDA-8B timing results, distinguish the fixed-template control from DCCD, and clarify the distribution induced by DGCD.

Question 1: What accuracy does the same AR model reach if it drafts and then reformats into JSON with no grammar constraint?

Response: We evaluated a two-stage AR baseline in which the same Qwen2.5 checkpoint first generates a free-form solution and then receives the original input, its draft, and the target schema to produce a JSON response. The second stage uses no grammar-based decoding constraints.

ModelFree-form AccuracyValid JSONValid & Correct
Qwen2.5-1.5B70.7%58.8%41.9%
Qwen2.5-3B83.9%24.1%20.9%
Qwen2.5-7B83.8%62.9%51.3%

Although the free-form drafts achieve 70.7–83.9% accuracy, only 20.9–51.3% of the reformatted outputs are both valid and correct. In particular, the 3B model has the highest draft accuracy among these configurations, but only 24.1% of its reformatted outputs are valid JSON. This illustrates that strong draft reasoning does not, by itself, ensure reliable, structured output from a second, unconstrained AR pass.

These results show that separating drafting from formatting, with structure requested through prompting alone, is insufficient to reliably produce valid and correct outputs in this evaluation. They motivate explicit structural enforcement, while the matched-template comparison in Q2 addresses the separate question of whether that enforcement must use diffusion. We will include this baseline alongside the constrained methods in the revised version.

Q2: What accuracy does draft-conditioned AR infilling reach on the same fixed template with the same slot caps?

We thank the reviewer for this question. Draft-conditioned AR infilling with a fixed template effectively reduces to constrained autoregressive decoding, where the model must generate content under predefined structural constraints. Imposing such constraints introduces a distribution shift, referred to as the projection tax, in DCCD [A], which can degrade generation accuracy.

A two-stage approach consisting of unconstrained AR drafting followed by constrained AR structuring addresses this issue partially, but remains subject to the limitations of AR constrained decoding. In particular, AR decoding enforces constraints sequentially and locally, without the ability to jointly revise previously generated content. This is closely related to the DCCD baseline considered in our evaluation.

In contrast, our DGCD uses diffusion-based infilling over a fixed structural template, enabling iterative refinement of content across multiple positions rather than committing to a strictly left-to-right generation order. Importantly, this also allows DGCD to adapt its generation process to predefined slot-level token budgets, reducing the risk of incomplete content under tight constraints.

Thus, our primary contribution is to introduce diffusion-based, budget-aware infilling that supports non-autoregressive refinement within a globally fixed structure.

[A] Reddy, Avinash, et al. "The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding." ICML 2026.

AuthorsOpenReview ↗

Response to Reviewer nukg [Part II]

Question 3: What is DGCD's latency with LLaDA-8B, the structurer used for the accuracy results?

Response: We thank the reviewer for raising this question. Figure 7 reports the end-to-end latency using our fine-tuned 1.5B diffusion structurer. To address the reviewer's concern, we additionally report the latency for LLaDA-8B, the structurer used in our accuracy evaluations, with 32 denoising steps.

MethodLatency (min)
Unconstrained AR0.256
CD1.163
DCCD1.470
DGCD (1.5B)0.409
DGCD (LLaDA-8B)0.866

DGCD with LLaDA-8B achieves an average end-to-end latency of 0.866 minutes per response, of which 0.610 minutes are spent on diffusion-based structuring and 0.256 minutes on AR drafting. This remains 1.34× faster than CD and 1.70× faster than DCCD under our experimental setting. We emphasize that latency depends on the number of denoising steps, sequence lengths, batching, and hardware utilization; all reported values correspond to our experimental configuration.

The latency advantage stems from the parallel generation mechanism of diffusion models, which allows multiple content positions to be generated and refined simultaneously, rather than strictly token by token as in AR decoding. Consequently, diffusion-based structuring latency is primarily governed by the number of denoising steps rather than the number of generated tokens. By conditioning on an already-computed AR draft, DGCD can produce structurally valid responses with relatively few denoising steps, reducing the computational overhead of the structuring stage.

Furthermore, both the fine-tuned 1.5B structurer and the untuned LLaDA-8B structurer achieve the same structural faithfulness, preserving the original AR draft's answer without modification. This demonstrates that a smaller, specialized diffusion model can provide reliable schema-constrained generation at substantially lower latency. We therefore use the 1.5B structurer in our efficiency experiments to demonstrate the potential for low-latency, structurally faithful generation in a composite AR–diffusion architecture, while retaining LLaDA-8B for the primary accuracy evaluations.

We will clarify the distinction between the 1.5B and 8B structurers in the revised manuscript and explicitly report their respective latency configurations.

Question 4: What distribution does DGCD's stage 2 sample from, and how does its deviation from the draft-conditioned, validity-conditioned dLLM distribution compare with Eq. 8 for CD?

Response: The reviewer is correct that DGCD masks and renormalizes content-token distributions. Fixing structural tokens does not make this operation distribution-preserving, nor does independent sampling at several positions imply sampling from a joint conditional over those positions. We will make these distinctions explicit.

Distribution induced by the sampler

Condition on the input xx, draft yy, and instantiated template σ\sigma. Let sns_n be the partially filled canvas at reverse step nn, and let Rn(sn)R_n(s_n) be the positions selected for unmasking by the sampler. For the categorical sampler described in Algorithm 1, with a deterministic selection rule and tie-breaking convention, define

\frac{ \pi_{\phi,i}(a\mid s_n,x,y)\mathbf{1}[a\in A_i] }{ \alpha_{n,i}(s_n,x,y) },\qquad \alpha_{n,i}(s_n,x,y)= \sum_{v\in A_i}\pi_{\phi,i}(v\mid s_n,x,y).$$ Assuming the normalizers are positive, the transition kernel is $$K_n(s_{n-1}\mid s_n,x,y) =\mathbf{1}\!\left[ (s_{n-1})_{\overline{R_n}}=(s_n)_{\overline{R_n}} \right] \prod_{i\in R_n(s_n)} q_{n,i}\!\left((s_{n-1})_i\mid s_n,x,y\right).$$ Starting from the fixed template canvas $s_N=\sigma$, the structured-output distribution is $$q_{\mathrm{DGCD}}(z\mid x,y,\sigma) =\sum_{\substack{s_{N-1},\ldots,s_0\\ \operatorname{stripPAD}(s_0)=z}} \prod_{n=N}^{1} K_n(s_{n-1}\mid s_n,x,y).$$ This expression includes the state-dependent reveal order and independent sampling within each selected block. Randomness in position selection, if used, must additionally be included in the transition kernel. The end-to-end distribution also averages over the draft distribution and, where applicable, the template-selection procedure.
AuthorsOpenReview ↗

Response to Reviewer nukg [Part III]

Comparison with Eq. 8 for CD

For a direct comparison, consider a reference sampler that starts from the same template and uses the same state-dependent position-selection rule, but omits content-token restrictions. Let P0(τ∣x,y,σ)P_0(\tau\mid x,y,\sigma) be its probability of a decoding trajectory τ\tau, and let C\mathcal C denote trajectories whose revealed tokens satisfy the restrictions. Then

P_0(\tau\mid x,y,\sigma)\, \mathbf{1}[\tau\in\mathcal C] \prod_{n=N}^{1} \prod_{i\in R_n(s_n)} \alpha_{n,i}(s_n,x,y)^{-1}.$$ Thus, content restriction introduces a **trajectory-dependent reweighting**, analogous to the prefix-dependent reweighting in Eq. 8. In contrast, conditioning the reference trajectory distribution on $\mathcal C$ multiplies every admissible trajectory by the same normalizing constant. These trajectory laws coincide only when the cumulative normalizer is constant over admissible trajectories, subject to matching support. The distinction from AR constrained decoding is therefore specific: **structural tokens are supplied as context instead of being generated through constrained next-token sampling**. Content restrictions remain, and parallel sampling introduces a further distinction from joint sampling.
Reviewer_fjBLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper combined an autoregressive large language model as a problem solver and a diffusion language model as a structured output rewritter as a two-stage system for higher problem-solving accuracy and stronger output structure adherence. Starting from the observation that AR LLMs are good at solving problems in free form but degrade for constrained decoding, while dLLMs are good at structured generation but incompetent at solving problems, the authors propose a drafting-first, then structured-generating approach to make use of the advantages of both LMs. On math reasoning and agentic benchmarks, the authors have shown that their proposed method is better than the CD and DCCD baselines. For analytical experiments, DGCD is also considered to be more parameter-efficient and low-latency than existing approaches.

Critical strengths
  • This work is training-free and therefore easy to implement and apply, and is flexible for future modification based on certain application scenarios.
  • The authors show low latency and high accuracy for structured generation, which are critical advantages for real-world agentic environments.
  • The paper finds a balance between AR and dLLM to maximize their potential while compensating for each other's disadvantages.
Critical weaknesses
  • The proposed method, DGCD, requires a schema-to-template conversion. However, there is no general algorithm provided in this paper that converts an arbitrary schema with complex constraints into a template that suffices for DGCD.
  • All template slots have a fixed size capacity bb's once decoding starts. However, the paper does not discuss how this parameter is determined after the draft and before the structured generation, if bb's are not predefined hyperparameters.
  • The authors strengthen one of the advantages of DGCD by claiming its ~0% performance drop in Figure 5, while in Section 4.2, they explicitly mention that the drop is, in fact, 1.6 points (65.2 → 63.6). For academic rigor and consistency, it is better to report the performance degradation of DGCD as nonzero for fairness and clarity.
  • There is no direct reliability analysis of how semantically faithful dLLMs can convert AR LLMs' output in free form into structured output.
Questions for authors
  • Can the authors show the algorithm that is used to convert schema to template, and explicitly how it deals with common constraints like: 1) data format restriction, 2) cross-field restriction, 3) nested objects?
  • Can the authors explain how to determine the capacity for each slot in the template before the dLLM's decoding process?
  • One important potential of dLLMs is revising their output by remasking and regeneration during the denoising process. Is it possible to enable DGCD to make use of this feature to further strengthen its capability?
4 comments
AuthorsOpenReview ↗

Response to Reviewer fjBL [Part I]

We thank the reviewer for recognizing the benefits of combining an autoregressive problem solver with a diffusion-based structurer. Your questions highlight implementation details and evaluation distinctions that deserve clearer treatment. We address the questions and corresponding weaknesses together below, including additional experiments and a correction to Figure 5.

Question 1: Can the authors show the algorithm that is used to convert schema to template, and explicitly how it deals with common constraints like:

  1. Data format restriction
  2. Cross-field restriction
  3. Nested objects

Response to Question 1: We thank the reviewer for highlighting this point. We agree that the schema-to-template construction needs a more explicit description. DGCD targets template-expressible schemas, as stated in Section 2; it does not assume that arbitrary semantic constraints can be enforced by fixing structural tokens alone. Below, we describe the construction and the additional decoding restrictions used for the constraint types raised by the reviewer.

Schema-to-template conversion

Given a schema represented as a Pydantic model, slot capacities, and supported constraint specifications, the construction proceeds as follows:

Inputs: Schema S, slot capacities B, supported constraints C

1. Determine array sizes:
   use specified sizes when fixed; otherwise predict an item count
   within the schema-defined bounds.

2. Recursively traverse the resulting object and array structure,
   preserving field names, delimiters, and separators as fixed tokens.

3. Replace each leaf value with a masked slot of the appropriate
   type and capacity.

4. Associate each slot with its format and value restrictions,
   and record supported dependencies between fields.

Outputs: Template, slot constraints, and field-dependency graph.

During generation, structural tokens remain fixed. Content slots are filled subject to their associated restrictions; for supported acyclic field dependencies, prerequisite fields are decoded before their dependent fields. Unused padding positions are removed when reconstructing the final JSON output.

Handling the three constraints

The three constraint classes raised by the reviewer are handled as follows:

(1) Data-format restrictions. Fixed-format values are represented using fixed separators and constrained masked positions. For example, a date is represented as:

"MMMM-MM-MM"  (YYYY-MM-DD)

The hyphens are fixed, and masked positions are restricted to valid calendar dates. At each denoising step, candidate dates inconsistent with already generated digits are eliminated. Thus, invalid dates such as 2023-02-30 cannot be produced, irrespective of the order in which tokens are unmasked.

(2) Cross-field restrictions. We represent dependencies as a directed graph and decode dependent fields in topological order. For example, to enforce effective_date ≤ expiration_date, we first decode effective_date and subsequently restrict expiration_date to dates greater than or equal to it. The same mechanism handles categorical dependencies, such as restricting location_type based on the generated incident_type.

(3) Nested objects and arrays. Nested objects are handled through recursive schema traversal. Their structural tokens are fixed, while leaf values become masked slots. For variable-length arrays, we employ count-first decoding: first predict the number of elements within schema-defined bounds, then instantiate that many item templates and decode their fields. This avoids forcing the model to populate non-existent array elements.

Additional empirical evaluation

We evaluated these constraints on 30 insurance claims using LLaDA-8B, with the same Qwen2.5-7B draft for each claim across configurations.

ConfigurationField accuracyDate accuracyRule violations
Unconstrained slots0.4050.63613/90
+ Date constraints0.5820.74212/90
+ Cross-field and enum constraints0.6470.7330/90

The full constraint configuration improved overall field accuracy from 0.405 to 0.647 and reduced observed rule violations from 13/90 to 0/90. Separately, for variable-length arrays, count-first decoding increased the number of outputs with the correct array size from 2/30 to 24/30 and strict field accuracy from 0.471 to 0.610, compared with a fixed two-element template.

These results demonstrate support for the evaluated format, dependency, and nested-structure constraints. We will include the construction, examples, and explicit scope of supported constraints in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer fjBL [Part II]

Question 2: Can the authors explain how to determine the capacity for each slot in the template before the dLLM's decoding process?

Response to Question 2: We thank the reviewer for this question. Slot capacities are predefined, task-dependent hyperparameters in the reported experiments. They are fixed before diffusion decoding and need not equal the exact length of the desired response. Their purpose is to allocate sufficient space for each field within the overall output budget; unused positions can be removed through padding.

For example, our GSM8K experiment allocates three reasoning slots of 64 tokens each and a final-answer slot of 32 tokens. This provides 224 content-token positions, in addition to the fixed structural tokens. We do not claim an optimal capacity-selection procedure: an insufficient allocation can limit how much task-relevant content the output can express.

Generation within fixed slot capacities

To evaluate whether DGCD can produce useful, complete responses within this allocation, we tested all 1,319 GSM8K test problems, using Qwen2.5-7B-Instruct as the AR model and LLaDA-8B-Instruct as the dLLM.

ApproachOutputs fitting budgetSemantically complete responsesGSM8K accuracy
AR alone7.0%—82.5%
dLLM alone100%81.3%37.8%
DGCD (AR → dLLM)100%96.9%80.5%

AR-alone responses were generated without a length constraint and subsequently checked against the template budget. Semantic completeness measures whether the reasoning-slot statements are complete rather than truncated mid-sentence or mid-reasoning.

Budget compliance follows from the fixed allocation. The substantive result is that DGCD achieves 80.5% accuracy within that allocation, compared with 82.5% for unrestricted AR generation, while increasing semantic completeness from 81.3% to 96.9% relative to standalone dLLM generation. These results suggest that draft guidance helps the diffusion model express a solution within the available slots while substantially retaining task accuracy.

Relation to semantic fidelity

We agree that semantic completeness and faithful preservation of the draft are distinct. A response can contain complete statements while changing the draft's answer or omitting relevant content. Likewise, a small aggregate accuracy difference can conceal correct-to-incorrect and incorrect-to-correct transitions that offset each other.

Accordingly, we interpret the results above as evidence of budget compliance, completeness, and task-level accuracy, rather than a direct measurement of full semantic fidelity. DGCD does not guarantee preservation of every statement in the draft. We will make this distinction explicit in the revised manuscript; a direct fidelity assessment requires paired comparisons of the draft and structured output.

Correction to Figure 5

The reviewer is correct that the “~0%” annotation is inaccurate. DGCD's MATH500 accuracy decreases from 65.2% to 63.6%, a nonzero loss of 1.6 percentage points. We will correct Figure 5 to consistently report net accuracy loss, using the following values:

MethodReference accuracyStructured accuracyNet loss
CD65.2%42.4%22.8 points
DCCD65.2%52.8%12.4 points
Constrained diffusion18.4%18.2%0.2 points
DGCD65.2%63.6%1.6 points

The reference is unconstrained Qwen2.5-7B-Instruct for CD, DCCD, and DGCD, and unconstrained LLaDA-8B-Instruct for standalone constrained diffusion.

We will also revise the axis, caption, and accompanying discussion to distinguish net accuracy loss from the correct-to-incorrect transition rate. The supported conclusion is that DGCD substantially reduces the measured accuracy loss from structuring, not that it eliminates that loss.

AuthorsOpenReview ↗

Response to Reviewer fjBL [Part III]

Question 3: One important potential of dLLMs is revising their output by remasking and regeneration during the denoising process. Is it possible to enable DGCD to make use of this feature to further strengthen its capability?

Response to Question 3: Yes. DGCD could incorporate remasking by keeping structural tokens fixed while selectively remasking and regenerating content positions, conditioned on the original input and draft. The current Algorithm 1 does not revisit revealed positions; this extension would allow previously generated content to be revised.

For example, a refinement pass could revisit low-confidence content or fields flagged by a consistency check. The same slot restrictions would apply during regeneration. If a revised field affects another field, the corresponding dependency constraints would also need to be rechecked. The slot capacities would remain unchanged, preserving the output-budget bound.

Existing approaches such as ReMDM provide a relevant starting point for incorporating remasking into pretrained masked diffusion models [1]. We have not evaluated this extension within DGCD, so we do not claim an additional accuracy improvement. Its effect on task accuracy, draft fidelity, and inference latency is a useful direction for further evaluation.

Reference

[1] Wang et al., Remasking Discrete Diffusion Models with Inference-Time Scaling.

AuthorsOpenReview ↗

Urgent request to reviewer

Thank you again for your thoughtful review and constructive feedback.

As the reviewer discussion phase approaches its end, we would like to kindly follow up on our rebuttal. We have carefully addressed your concerns and added new experiments.

We would be very grateful if you could review our responses. We sincerely appreciate your time, efforts, and service to the research community.

Regards,

Authors

Reviewer_5KZdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Constrained decoding in autoregressive generation might worsen the model's downstream performance due to changes in the generation distribution and unawareness of budget control. Prior work partially solved these problems by generating an unconstrained draft before constrained decoding, but these problems can still persist.

In this paper, the authors instead use diffusion language models (dLLM) to generate the complete, schema-valid outputs. The authors focus on constraints that can be expressed as a fixed template, and they fix these template words in dLLMs before generating the complete outputs so the complete outputs will always conform to the template and does not go out of token budgets.

Critical strengths
  • Novel idea to combine an autoregressive drafter and a template-constrained diffusion model.
  • Clear performance advantage over prior work.
Critical weaknesses
  • Only template-based constraints (with explicit token counts for each slot) can be used in this work. For many tasks, such templates are not available or just cover part of the constraint. For example, if a JSON schema requires to fill in a string of a date, this method would have to simply use string for such constraint.
Questions for authors

These are just minor issues. The critical weakness is the major issue.

  1. The experiments in Figure 8 (comparing accuracy per billion parameters) are unacceptable, because performance does not improve linearly to parameter counts. Can the authors provide complete results so that we can see the whole picture directly?
  2. Why do you train a new diffusion model in Appendix G instead of using available models?
  3. In Figure 7, I want to see the wall-clock time for each component in these methods. For example, for DCCD, I want to see the time to generate the draft and the time to generate the final output seperately.
  4. In Figure 7, why do you use your home-trained diffusion model instead of LLaDA used in previous experiments? I want to see the results on the previous setups. Also, I want to see the raw results (not averaged over several different drafters).
3 comments
AuthorsOpenReview ↗

Response to Reviewer 5KZd [Part I]

We thank the reviewer for the positive assessment and helpful suggestions. Below, we clarify the supported constraints and provide additional accuracy and timing results.

Scope of supported constraints

Only template-based constraints with explicit slot capacities can be used. Such templates may cover only part of a schema, for example when a string field must contain a date.

We agree that fixing the output skeleton is not sufficient to enforce every requirement of a schema. In DGCD, the template specifies where fields appear, while additional restrictions determine which values are admissible within those fields. Our implementation uses fixed templates, but the values within each slot can also be constrained.

For example, a date field can restrict its admissible completions to calendar-valid values in the format YYYY-MM-DD, accounting for month lengths and leap years. If the committed year and month are 2023-02, the remaining day must be between 01 and 28. Our constraint-validation test over 3,000 random unmasking orders produced no invalid dates. We will add the implementation details, including how the decoder checks that newly revealed tokens remain compatible with a valid date.

To examine the scope beyond the original templates, we additionally evaluated draft conditioning with the finite-automata constrained diffusion decoder of Dang and Ermon [B]. That method supports constraints expressible as finite automata. This provides a concrete way to test whether the benefit of draft guidance carries over to another constraint-enforcement mechanism.

We used its LLaDA-8B decoder with the same base prompt, automaton, decoding settings, and evaluation procedure for both conditions, adding an AR draft in the second condition.

BenchmarkAutomaton-constrained decoder, no AR draftSame decoder with AR draftAccuracy gain
BFCL Simple92.0%95.5%+3.5 points
BFCL Parallel83.5%89.0%+5.5 points
JEVBench, all tiers69.7%74.5%+4.8 points
JEVBench, hard tier40.5%54.0%+13.5 points

Draft conditioning improves accuracy in each evaluated setting, with the largest gain on the JEVBench hard tier. These results show that draft guidance also improves accuracy with the automaton-constrained decoder.

Fixed slot capacities remain a limitation of our original implementation. We will clarify this scope and report the automaton-based experiment separately.

[B] M. Dang and S. Ermon. Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata. arXiv:2607.07026, 2026.

Response to Questions

Question 1: The experiments in Figure 8 (comparing accuracy per billion parameters) are unacceptable, because performance does not improve linearly to parameter counts. Can the authors provide complete results so that we can see the whole picture directly?

Response: We agree that raw accuracy, explicit model identities, and total parameter counts provide a clearer comparison than the normalized ratio in Figure 8. We will replace the ratio-based presentation accordingly.

We will also distinguish two experimental settings more clearly. The main accuracy results use LLaDA-8B for DGCD, while DCCD reuses the AR drafter. The compact-model experiment in Figure 8 instead uses a 1.5B second-stage model for each method. Results from these settings should not be combined.

For clarity, the following table gives the main accuracy results for the drafter configurations discussed here. These are the LLaDA-8B DGCD results from Table 2, rather than measurements of the compact 1.5B diffusion structurer.

AR DrafterMethodGSM8KMATH500Financial EntitiesPII
1.5BDCCD71.638.810.32.0
1.5BDGCD71.652.027.66.0
3BDCCD82.247.410.80.0
3BDGCD83.966.249.68.0
7BDCCD84.052.824.22.0
7BDGCD84.163.663.616.0
8BDCCD84.537.217.82.0
8BDGCD83.645.268.810.0

For example, on MATH500, DGCD with a 3B AR drafter achieves 66.2% accuracy, exceeding DCCD with a 7B AR drafter (52.8%), despite using a smaller total model configuration. Similarly, on Financial Entity Extraction, DGCD with a 3B drafter achieves 49.6% accuracy, compared with 24.2% for DCCD with a 7B drafter.

We will report raw accuracy, both checkpoint identities, and total unique parameters for each configuration.

AuthorsOpenReview ↗

Response to Reviewer 5KZd [Part II]

Question 2: Why do you train a new diffusion model in Appendix G instead of using available models?

Response: The main experiments already use the publicly available LLaDA-8B-Instruct, so training a new diffusion model is not a prerequisite for DGCD. The compact model in Appendix G addresses an additional question: whether a smaller structurer, specialized to generating structured outputs from a draft, can reduce inference cost.

We trained the 1.5B checkpoint to test whether structuring an existing draft could be handled more efficiently by a smaller model. We therefore converted and trained a 1.5B checkpoint for the separate efficiency experiment.

Its reported average latency is 0.409 minutes per response, compared with 0.866 minutes for the LLaDA-8B configuration. These timings motivate studying the compact alternative, but do not by themselves establish equal task accuracy or answer preservation. We will report its corresponding accuracy alongside its latency in the compact-model results and clearly separate those results from the main LLaDA-8B evaluation.

Question 3: In Figure 7, I want to see the wall-clock time for each component in these methods. For example, for DCCD, I want to see the time to generate the draft and the time to generate the final output seperately.

Response: We agree that a stage-level breakdown makes the computational costs easier to assess. The following table separates the shared drafting time from the remaining structuring cost on MATH500 and reports each drafter individually.

AR DrafterUnconstrained (min)CD (min)DCCD (min)DGCD (LLaDA-8B) (min)
Llama-3.2-1B0.2940.9991.3580.904
Qwen2.5-1.5B0.3760.8921.2980.986
Qwen2.5-3B0.2051.1901.4450.815
Qwen2.5-7B0.1481.5731.7810.758
Average0.2561.1641.4710.866

For DGCD using LLaDA-8B, the average AR drafting time is 0.256 minutes, while diffusion-based structuring with 32 denoising steps requires approximately 0.610 minutes, yielding an average total latency of 0.866 minutes.

We will revise Figure 7 to distinguish drafting and structuring latency explicitly and provide the corresponding per-drafter results.

Question 4: In Figure 7, why do you use your home-trained diffusion model instead of LLaDA used in previous experiments? I want to see the results on the previous setups. Also, I want to see the raw results (not averaged over several different drafters).

Response: We thank the reviewer for highlighting this inconsistency in the presentation. Figure 7 uses our fine-tuned 1.5B diffusion structurer, whereas the primary accuracy experiments use the publicly available LLaDA-8B. Our intention was to demonstrate the additional inference-efficiency benefits obtainable from a smaller, task-specialized diffusion structurer.

To address the reviewer's concern, we additionally report the latency of DGCD using LLaDA-8B with 32 denoising steps, consistent with the structurer used in our main accuracy experiments.

MethodAverage Latency (min)
Unconstrained AR0.256
CD1.163
DCCD1.470
DGCD (1.5B)0.409
DGCD (LLaDA-8B)0.866

DGCD with LLaDA-8B achieves an average end-to-end latency of 0.866 minutes per response, of which 0.610 minutes are spent on diffusion-based structuring. Even with the larger structurer, DGCD remains 1.34× faster than CD and 1.70× faster than DCCD under our experimental configuration.

The latency advantage arises from the parallel generation mechanism of diffusion models. Unlike autoregressive constrained decoding, which generates tokens sequentially, diffusion-based structuring allows multiple content positions to be generated and refined simultaneously. Consequently, its inference cost depends strongly on the number of denoising steps rather than requiring one sequential decoding step per generated token.

By conditioning on an already-computed AR draft, DGCD can produce structurally valid responses with relatively few denoising steps, reducing the overhead associated with the structuring stage.

We will incorporate the per-drafter results above into Figure 7 and identify the checkpoint and timing settings for each configuration.

Thank you for your replies.

I think the main strength of this paper to me is the idea of combining unconstrained autoregressive drafter and constrained diffusion model. Although this sounds novel, it is not novel enough for a direct score of 4.

The authors addressed the main issue of limitations of template-based constraints (W1) in the response, and showed that this method might be possible to be combined with other constrained decoding methods developed for diffusion models. However, I think this goes beyond the original scope of this paper, because the motivation of this paper are explicitly centered around the benefits of using templates as constraints in diffusion models.

I thank the authors for addressing the soundness issues in the paper, but I think even at its best status, I can still only give a score of 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Urgent Request to AC

Dear AC,

Thank you for your time and for handling our paper. We wanted to request if you could ask the reviewers to respond to our rebuttals. We have only received one response from the reviewers so far, and we have addressed their concerns. Since the deadline is approaching, this would greatly help with further evaluation of our work.

Regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.