- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose DGCD for schema-constrained generation: an AR model writes an unconstrained draft, and a masked diffusion LM then fills the content slots of a template whose structural tokens are fixed in advance, conditioned on the input and the draft. On GSM8K, MATH500, and two extraction benchmarks with six AR drafters (1B–14B), they report 100% schema adherence and higher accuracy than constrained decoding and DCCD, robustness to tight token budgets, and lower latency.
Critical strengths
- The method target a known problem with a well-motivated connection. The paper’s insight is that solving and formatting need not share a generation mechanism: the AR model solves freely, and schema infilling is offloaded to a dLLM that conditions on structure fixed in advance.
- Results cover six AR drafters from two model families (1B–14B) on four benchmarks spanning reasoning and extraction. The direction of the effect is consistent across this grid
Critical weaknesses
- The accuracy gain is not isolated to the proposed mechanism. DGCD differs from CD by adding a draft pass, and from DCCD by changing the formatter model, fixing the template, and using diffusion, all at once. The paper attributes the gain to diffusion, but the needed controls are absent: -- A draft followed by unconstrained reformatting with the same AR model, -- Draft-conditioned AR infilling of the same fixed template with the same slot caps
- In Table 2, DCCD reuses the drafter as its formatter, while DGCD always adds LLaDA-8B. The “1B” DGCD row is a 9B system compared with a 1B one. The structurer also sees the original input, so on extraction it may be solving the task itself.
- Accuracy is reported with LLaDA-8B, but latency and parameter efficiency use a fine-tuned 1.5B structurer whose accuracy is not reported.
- Appendix H formalizes the distortion of AR constrained decoding but gives no corresponding analysis for DGCD, which applies the same mask-and-renormalize step at content positions (Eq. 3) and additionally samples multiple positions per step from per-position marginals. The paper does not state what distribution DGCD samples from, nor give conditions under which the draft’s answer is preserved.
Questions for authors
- What accuracy does the same AR model reach if it drafts and then reformats into JSON with no grammar constraint?
- What accuracy does draft-conditioned AR infilling reach on the same fixed template with the same slot caps?
- What is DGCD’s latency with LLaDA-8B, the structurer used for the accuracy results?
- What distribution does DGCD’s stage 2 sample from, and how does its deviation from the draft-conditioned, validity-conditioned dLLM distribution compare with Eq. 8 for CD?
5 comments
Response to Reviewer nukg [Part II]
Question 3: What is DGCD's latency with LLaDA-8B, the structurer used for the accuracy results?
Response: We thank the reviewer for raising this question. Figure 7 reports the end-to-end latency using our fine-tuned 1.5B diffusion structurer. To address the reviewer's concern, we additionally report the latency for LLaDA-8B, the structurer used in our accuracy evaluations, with 32 denoising steps.
| Method | Latency (min) |
|---|---|
| Unconstrained AR | 0.256 |
| CD | 1.163 |
| DCCD | 1.470 |
| DGCD (1.5B) | 0.409 |
| DGCD (LLaDA-8B) | 0.866 |
DGCD with LLaDA-8B achieves an average end-to-end latency of 0.866 minutes per response, of which 0.610 minutes are spent on diffusion-based structuring and 0.256 minutes on AR drafting. This remains 1.34× faster than CD and 1.70× faster than DCCD under our experimental setting. We emphasize that latency depends on the number of denoising steps, sequence lengths, batching, and hardware utilization; all reported values correspond to our experimental configuration.
The latency advantage stems from the parallel generation mechanism of diffusion models, which allows multiple content positions to be generated and refined simultaneously, rather than strictly token by token as in AR decoding. Consequently, diffusion-based structuring latency is primarily governed by the number of denoising steps rather than the number of generated tokens. By conditioning on an already-computed AR draft, DGCD can produce structurally valid responses with relatively few denoising steps, reducing the computational overhead of the structuring stage.
Furthermore, both the fine-tuned 1.5B structurer and the untuned LLaDA-8B structurer achieve the same structural faithfulness, preserving the original AR draft's answer without modification. This demonstrates that a smaller, specialized diffusion model can provide reliable schema-constrained generation at substantially lower latency. We therefore use the 1.5B structurer in our efficiency experiments to demonstrate the potential for low-latency, structurally faithful generation in a composite AR–diffusion architecture, while retaining LLaDA-8B for the primary accuracy evaluations.
We will clarify the distinction between the 1.5B and 8B structurers in the revised manuscript and explicitly report their respective latency configurations.
Question 4: What distribution does DGCD's stage 2 sample from, and how does its deviation from the draft-conditioned, validity-conditioned dLLM distribution compare with Eq. 8 for CD?
Response: The reviewer is correct that DGCD masks and renormalizes content-token distributions. Fixing structural tokens does not make this operation distribution-preserving, nor does independent sampling at several positions imply sampling from a joint conditional over those positions. We will make these distinctions explicit.
Distribution induced by the sampler
Condition on the input , draft , and instantiated template . Let be the partially filled canvas at reverse step , and let be the positions selected for unmasking by the sampler. For the categorical sampler described in Algorithm 1, with a deterministic selection rule and tie-breaking convention, define
\frac{ \pi_{\phi,i}(a\mid s_n,x,y)\mathbf{1}[a\in A_i] }{ \alpha_{n,i}(s_n,x,y) },\qquad \alpha_{n,i}(s_n,x,y)= \sum_{v\in A_i}\pi_{\phi,i}(v\mid s_n,x,y).$$ Assuming the normalizers are positive, the transition kernel is $$K_n(s_{n-1}\mid s_n,x,y) =\mathbf{1}\!\left[ (s_{n-1})_{\overline{R_n}}=(s_n)_{\overline{R_n}} \right] \prod_{i\in R_n(s_n)} q_{n,i}\!\left((s_{n-1})_i\mid s_n,x,y\right).$$ Starting from the fixed template canvas $s_N=\sigma$, the structured-output distribution is $$q_{\mathrm{DGCD}}(z\mid x,y,\sigma) =\sum_{\substack{s_{N-1},\ldots,s_0\\ \operatorname{stripPAD}(s_0)=z}} \prod_{n=N}^{1} K_n(s_{n-1}\mid s_n,x,y).$$ This expression includes the state-dependent reveal order and independent sampling within each selected block. Randomness in position selection, if used, must additionally be included in the transition kernel. The end-to-end distribution also averages over the draft distribution and, where applicable, the template-selection procedure.Response to Reviewer nukg [Part III]
Comparison with Eq. 8 for CD
For a direct comparison, consider a reference sampler that starts from the same template and uses the same state-dependent position-selection rule, but omits content-token restrictions. Let be its probability of a decoding trajectory , and let denote trajectories whose revealed tokens satisfy the restrictions. Then
P_0(\tau\mid x,y,\sigma)\, \mathbf{1}[\tau\in\mathcal C] \prod_{n=N}^{1} \prod_{i\in R_n(s_n)} \alpha_{n,i}(s_n,x,y)^{-1}.$$ Thus, content restriction introduces a **trajectory-dependent reweighting**, analogous to the prefix-dependent reweighting in Eq. 8. In contrast, conditioning the reference trajectory distribution on $\mathcal C$ multiplies every admissible trajectory by the same normalizing constant. These trajectory laws coincide only when the cumulative normalizer is constant over admissible trajectories, subject to matching support. The distinction from AR constrained decoding is therefore specific: **structural tokens are supplied as context instead of being generated through constrained next-token sampling**. Content restrictions remain, and parallel sampling introduces a further distinction from joint sampling.
Response to Reviewer nukg [Part I]
Response to Reviewer nukg — Score 2/4
We thank the reviewer for identifying the controls needed to clarify the contributions of draft conditioning, template allocation, and the choice of structurer. Below, we report the additional AR-reformatting and LLaDA-8B timing results, distinguish the fixed-template control from DCCD, and clarify the distribution induced by DGCD.
Response: We evaluated a two-stage AR baseline in which the same Qwen2.5 checkpoint first generates a free-form solution and then receives the original input, its draft, and the target schema to produce a JSON response. The second stage uses no grammar-based decoding constraints.
Although the free-form drafts achieve 70.7–83.9% accuracy, only 20.9–51.3% of the reformatted outputs are both valid and correct. In particular, the 3B model has the highest draft accuracy among these configurations, but only 24.1% of its reformatted outputs are valid JSON. This illustrates that strong draft reasoning does not, by itself, ensure reliable, structured output from a second, unconstrained AR pass.
These results show that separating drafting from formatting, with structure requested through prompting alone, is insufficient to reliably produce valid and correct outputs in this evaluation. They motivate explicit structural enforcement, while the matched-template comparison in Q2 addresses the separate question of whether that enforcement must use diffusion. We will include this baseline alongside the constrained methods in the revised version.
Q2: What accuracy does draft-conditioned AR infilling reach on the same fixed template with the same slot caps?
We thank the reviewer for this question. Draft-conditioned AR infilling with a fixed template effectively reduces to constrained autoregressive decoding, where the model must generate content under predefined structural constraints. Imposing such constraints introduces a distribution shift, referred to as the projection tax, in DCCD [A], which can degrade generation accuracy.
A two-stage approach consisting of unconstrained AR drafting followed by constrained AR structuring addresses this issue partially, but remains subject to the limitations of AR constrained decoding. In particular, AR decoding enforces constraints sequentially and locally, without the ability to jointly revise previously generated content. This is closely related to the DCCD baseline considered in our evaluation.
In contrast, our DGCD uses diffusion-based infilling over a fixed structural template, enabling iterative refinement of content across multiple positions rather than committing to a strictly left-to-right generation order. Importantly, this also allows DGCD to adapt its generation process to predefined slot-level token budgets, reducing the risk of incomplete content under tight constraints.
Thus, our primary contribution is to introduce diffusion-based, budget-aware infilling that supports non-autoregressive refinement within a globally fixed structure.
[A] Reddy, Avinash, et al. "The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding." ICML 2026.