acceptodds
Under review as a conference paper at ICLR 2027

Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models

Abstract

Diffusion Language Models (DLMs) have recently become increasingly competitive with autoregressive (AR) models, and even outperform them on certain tasks. Unlike AR models, DLMs produce output through iterative denoising without a left-to-right order. To further improve the performance of DLMs, we introduce PDC (Prefix-Denoising Consistency), a test-time self-verification method for DLMs. PDC exploits a distinctive test-time signal in DLMs under prefix conditioned regeneration, correct trajectories are more stable and reproducible than incorrect ones. Concretely, given an initially generated sample, PDC splits the sentence at an intermediate position and regenerates the remaining tokens conditioned on the fixed prefix. Across mathematical reasoning and commonsense reasoning benchmarks, PDC consistently improves upon the initial sample, outperforms independent generations under a computational constrained comparison, and is robust to different unmasking strategies and parameter settings. These results highlight prefix-conditioned regeneration as an effective DLM-specific primitive for test-time verification.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Models, Test-Time Verification, Inference-Time Scaling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2K6cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces Prefix-Denoising Consistency (PDC), a test-time self-verification and scaling method tailored for DLMs, which is particularly important when standard low-temperature DLM decoding suffers from trajectory collapse during repeated full generations, making naive parallel self-consistency (majority voting over independent runs) ineffective because samples follow near-identical unmasking paths. PDC takes an initial complete output, preserves an exact prefix at specific keep rates (ρ∈{0.1,0.5,0.9}\rho \in \{0.1, 0.5, 0.9\}), remasks the remaining suffix, regenerates the text, and aggregates candidate answers via majority voting. The proposed method is tested across mathematical reasoning (e.g. GSM8K, MATH-500, SVAMP) and commonsense reasoning benchmarks (e.g. CSQA, SQA) using models like Dream-7B and the LLaDA family.

Critical strengths
  1. The paper effectively identifies and formalizes the "reproduction gap”, proving theoretically in Theorem 2.2 that method performance depends on this gap. Additionally, they demonstrate empirically that correct DLM trajectories are significantly more reproducible under prefix conditioning than incorrect ones (R+>R−R^+ > R^-), providing a solid foundation for the approach.

  2. Strong Compute-Efficiency Profile is a plus: PDC outperforms standard full-generation self-consistency (INIT x4) while consuming fewer total denoising steps (≈2.5T\approx 2.5T vs. 4T4T).

  3. Strong Robustness Validation is a plus: The paper provides thorough checks across alternative unmasking strategies, semi-autoregressive blockwise settings, and varying sampling temperatures (demonstrating robustness even under stochastic τ=1.0\tau = 1.0 decoding)

Critical weaknesses
  1. Unless I misunderstood the paper, the underlying mechanism heavily mirrors established prefix-consistency and prefix-consensus frameworks originally designed for autoregressive language models to improve self-consistency and reasoning reliability (e.g., Iwase et al., "Reliable Chain-of-Thought via Prefix Consistency" and Jindal et al., "The Path of Least Resistance: Guiding LLM Reasoning Trajectories with Prefix Consensus"). Translating this concept to discrete diffusion’s masked output window reads as a straightforward architectural adaptation of existing AR verification strategies. Beyond achieving a minor reduction in total denoising steps, it remains unclear what fundamental algorithmic advance this adaptation provides for diffusion models.

  2. I believe that the empirical results expose a critical limitation: improvements on non-math, commonsense reasoning benchmarks (such as CSQA and SQA) are largely negligible (+0.00 to +0.98 points), questioning whether the method generalizes to broader text domains or is strictly specialized for structured arithmetic tasks. While short outputs naturally limit trajectory exploration in AR models, this issue is exacerbated in DLMs. This occurs because DLM outputs are bound to a fixed length window LL and token-level remasking operates across this static span; forcing prefix-conditioned regeneration on short, non-derivational tasks strips away the sequence runway required for meaningful alternative trajectory divergence. Consequently, the method functions more as a specialized arithmetic-correction heuristic than a universal test-time scaling solution, raising the question of what additional techniques are required to overcome this inherent DLM limitation.

  3. Because PDC depends entirely on an initial generation to anchor the prefix, its overall success is tightly bounded by the structural soundness of the base model's initial setup. If the preliminary prefix contains fatal logical or arithmetic flaws, partial suffix remasking is structurally incapable of rescuing the trajectory, rendering the verification loop ineffective for fundamentally malformed initial runs.

Questions for authors

Q1: Given that PDC yields robust gains primarily on multi-step arithmetic datasets like GSM8K and MATH-500 while showing negligible improvements on commonsense reasoning benchmarks like CSQA and SQA, any scientific explanation of why this happens and how this can be improved?

Q2: In AR models, prefix-anchored methods benefit from long-form chain-of-thought outputs that provide ample token runway for trajectory divergence. In contrast, how do the authors reconcile the structural reality of discrete diffusion language models where generation is bound to a static length window LL with token-level remasking distributed across the entire span when applied to short-form or non-derivational tasks where prefix preservation strips away the necessary exploration runway?

Q3: In Section 2.2, the theoretical characterization of PDC's expected accuracy relies on a stylized binary answer space A={a∗,b}A = \{a^*, b\} and models the three keep-rate regenerations using V3(p)V_3(p) under conditional independence (Eq. 2–5). Given that discrete diffusion language model trajectories share a common structural backbone from the initial prefix y1:Lρprefixy_{1:L_\rho^{\text{prefix}}}, how do the authors justify the conditional independence assumption among Rρ1,Rρ2,Rρ3R_{\rho_1}, R_{\rho_2}, R_{\rho_3} in practice, and to what extent does correlation across these prefix-conditioned trajectories violate the bounds derived in Theorem 2.2?

Q4: In Appendix B (Table 4), it seems like certain model and dataset configurations (e.g., LLaDA-8B on MATH-500 at length 512, or specific settings where same-wrong persistence Rρ−R_\rho^- spikes) exhibit negative separation metrics (Δ=Rρ+−Rρ−\Delta = R_\rho^+ - R_\rho^- shown in red). Under what specific conditions do these failure modes occur where wrong answers become more reproducible than correct ones, and how does the majority voting mechanism recover when multiple keep rates fall into this negative-gap regime?

1 comment

We thank the reviewer for the detailed reading of the theory and the appendix. Additions are in red in the revised PDF: Appendix I (Tables 13-16), Appendix I.1 (Tables 17-18) and Appendix K (Limitations). Pairs "a / b" below are LLaDA-1.5 / LLaDA-8B.

1. What PDC adds over AR prefix-consistency (Table 13).

PDC builds on prefix consistency but uses prefix-conditioned suffix denoising to obtain alternative answers from a single completed DLM trajectory. Iwase et al. use prefix regeneration to assess candidate reliability, while Jindal et al. cluster short reasoning prefixes and prune non-dominant clusters. PDC instead regenerates from several prefix lengths of one initial output and votes only over the regenerated answers.

This distinction matters in our low-temperature setting. At temperature 0.2, four independent LLaDA generations give the same final answer on 95% of GSM8K problems, and INIT ×2 / ×3 / ×4 stay within 0.6 points of INIT across all ten LLaDA settings. We implemented PC-WMV on the same DLMs and benchmarks at a nominal budget of 2.5T; it stays within 1.1 points of INIT across these settings. On GSM8K, PC-WMV achieves 61.4 / 61.8, compared with 66.3 / 65.9 for PDC (Table 13). Depending on the keep rate, PDC’s regenerations move away from an initially wrong answer in 12–78% of GSM8K cases.

PDC also requires no exposed weights, logits, or intermediate denoising states: it uses the final text and an interface supporting fixed-prefix regeneration (Section 1).

2. Small gains on CSQA and SQA; Q1 (Table 15, Appendix K).

We agree that the gain is smaller in non-math tasks. For LLaDA at keep rate 0.5, the reproduction gap R+ minus R- is 67-73 points on GSM8K, 38 on MATH-500 and 34-50 on SVAMP, but 8-10 on CSQA and 7 on SQA. Theorem 2.2 ties the gain to this gap, so a small gap predicts a small gain: the commonsense results follow the theory rather than contradict it.

Mechanistically, a wrong commonsense answer is a belief error that the model reproduces from any prefix, whereas a wrong math answer is usually a local derivation error that a re-derivation avoids. PDC therefore targets derivational errors; Appendix K states this as a limitation.

3. Q2: fixed window and exploration room (Appendix I, "Output length and regeneration room").

We measured it. LLaDA outputs fill the 256-token window on GSM8K and MATH-500, so the 0.1 regeneration has on average 231 free positions and the 0.9 regeneration still 28-33. Dream uses on average 227 of 512 tokens on GSM8K and 62 of 128 on CSQA, so even its 0.9 regeneration has more free positions than the model used. Appendix K notes that fixed windows can constrain longer derivations. Thank you for your clarification.

4. Dependence on the initial generation (Appendix K).

We respectfully disagree that the initial generation tightly bounds PDC. A wrong initial answer is not locked in: on GSM8K the regeneration at keep rate 0.5 departs from it in 78% / 72% of cases (Table 15) and reaches the correct answer in 45% / 41%. Nor do most errors sit in the early setup: where a wrong arithmetic step is detectable, the first one lies after the 0.5 cut in 76% / 74% of wrong GSM8K outputs (Appendix I.1). The bound applies only when the early framing itself is wrong, which Appendix K states as a limitation.

5. Q3: conditional independence (remark "Scope of Theorem 2.2", Section 2.2; Table 15).

The theorem is a stylized model, now stated as such, and we checked both of its simplifications. On GSM8K the pairwise phi correlations between the correctness of the three regenerations, given a correct initial answer, are at most 0.19 / 0.14, and the product-form majority probabilities are within 2.5 points of their empirical values; on MATH-500 the correlations reach 0.25.

The larger discrepancy is the binary answer space: 8.0% / 5.7% of GSM8K and 15.2% / 12.0% of MATH-500 problems move from one wrong answer to another, which the theorem counts as corrections. It therefore predicts 74.5 / 71.9 on GSM8K against observed 66.3 / 65.9.

6. Q4: negative separations in Table 4.

One correction first: the MATH-500 / LLaDA-8B / L=512 row is positive at all three keep rates. The negative cells are SVAMP / LLaDA-8B / L=512 at 0.1 (-3.8), SQA / LLaDA-8B / L=256 at 0.1 (-3.3) and four cells below one point in magnitude at keep rate 0.9 on CSQA and SQA.

They arise (a) at degenerate lengths, where few answers are extractable and R- is computed over a handful of wrong answers, and (b) at keep rate 0.9 on multiple-choice tasks, where both rates approach 100% and the difference is within noise. The vote recovers because V3 is monotone in each coordinate, so one negative coordinate does not flip the sign when the others are large. The caption of Table 4 now says this, and Table 15 gives the conditioning-set sizes.

Reviewer_fbnbOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Prefix-Denoising Consistency to improve the diffusion models several downstream accuracy in test-time, although the comparison is limited in easy tasks and incomparable FLOPs.

Critical strengths

The proposed Prefix-Denoising Consistency improves the diffusion models several downstream accuracy in test-time.

Critical weaknesses
  1. The selected tasks are almost easy tasks and are overfitted by the tested models. This not only makes their conclusion about "correct trajectories are more stable" is trivial, but also limits the solidness of the proposed method. More difficult tasks should be selected to explore the PDC's capability. A curve plot with the model's original scores on the horizontal axis and the scores after adding PDC on the vertical axis is worth being compiled to demonstrate the effectiveness and generalizability of the method.

  2. The direct comparison between PDC and baseline is not fair, as PDC uses significantly more FLOPs compared to the baseline. A comparable FLOPs should be important.

  3. More baselines on test-time verification should be compared.

Questions for authors

See Weakness.

2 comments

We thank the reviewer for the comments. Additions are in red in the revised PDF: Appendix I (Tables 13-16, Figure 3), Appendix I.1 (Table 17), Appendix J and Appendix K (Limitations). Pairs "a / b" below are LLaDA-1.5 / LLaDA-8B.

1. Task difficulty and the requested plot (Table 16, Figure 3).

We respectfully disagree that the benchmarks are saturated for these models: initial accuracies in Table 1 are 25.7-46.0% on MATH-500, 60.6-82.0% on GSM8K and 65-70% on StrategyQA. We stayed in the regime where open 7-8B DLMs are usable; competition-level mathematics is beyond them, leaving nothing to measure. Appendix K now states this as a limitation.

Two additions address difficulty directly. Table 16 splits MATH-500 by difficulty level: gains concentrate on Levels 1-2 (Level 1: 58.1 -> 69.8 and 62.8 -> 69.8 for the two LLaDA models), while Levels 4-5 change by at most 1.6 points; on Level 5 the initial trace is usually wrong from the start, which regeneration cannot repair. Figure 3 (left) is the requested plot of PDC against INIT accuracy over the 45 model-dataset-length cells of Appendix C: PDC lies above the diagonal in 44 of 45 cells (mean +3.8 points; +1.6 over the 15 Table 1 cells).

On the word "trivial": the reproduction gap between correct and wrong answers (Table 15) varies by an order of magnitude across tasks, 34-73 points at keep rate 0.5 on the math tasks for LLaDA against 5-10 on the commonsense tasks, and the gain tracks the gap as Theorem 2.2 predicts. The revision states where the signal fails rather than presenting it.

2. Comparable FLOPs (Table 13).

We agree that it should be clarified and report budgets in denoising steps: each step is one forward pass over the prompt and the length-L window for both generation and regeneration, so steps are proportional to FLOPs. PDC costs 1 + 0.9 + 0.5 + 0.1 = 2.5T. Table 2 already compares with self-consistency over four generations (4T), which PDC beats in 13 of 15 settings.

Table 13 adds INIT x2 (2T), INIT x3 (3T) and the prefix-consistency method of Iwase et al. at exactly 2.5T: none exceeds INIT by more than 1.1 points, while PDC gains +5.7 / +5.2 on GSM8K. In summary, PDC outperforms simple majority voting given the same number of denoising steps.

Extra samples do not help because independent resampling collapses at temperature 0.2: four LLaDA generations give the same final answer on 95% of GSM8K problems. The useful way to spend budget is to change the conditioning, and one regeneration at keep rate 0.5 (1.5T) already gives +14.7 / +12.7 points on GSM8K and +5.4 / +5.4 on MATH-500 (Table 17).

3. More test-time verification baselines (Table 13, Appendix J).

The paper compares with three TiF variants, state-of-the-art for DLM test-time methods, and with self-consistency. We add the method of Iwase et al. at 2.5T and confidence selection, the best of four independent samples by mean token log-probability (4T).

Confidence selection gives 59.2 / 62.2 on a 500-problem GSM8K subset, 83.7 / 83.7 on SVAMP and 81.2 / 80.4 on CSQA, i.e., it equals INIT. It fails for the same reason as self-consistency: the four samples are nearly identical, so there is nothing to select among. ` If the reviewer has specific methods in mind, we are happy to add them.

Thank for the authors' response. As the authors only address my third concern, I would like to raise my score to weak rejection with higher confidence.

Reviewer_jqUpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Prefix-Denoising Consistency (PDC), a test-time method for diffusion language models (DLMs). Given an initial output, PDC fixes prefixes at three lengths, regenerates the remaining suffix through denoising, and selects an answer by voting over the regenerated outputs. The method is motivated by the observation that correct answers are more likely than incorrect answers to be reproduced under prefix-conditioned regeneration. Across three DLMs and five reasoning benchmarks, PDC improves over the initial output in 14 of 15 main settings, with one tie. At generation length 128, it outperforms voting over four independent generations in 13 of 15 settings while using fewer nominal denoising steps

Critical strengths

The method is simple, requires no additional training, and can be applied to existing DLMs.

The reproduction-rate asymmetry is an interesting empirical signal. Several improvements, particularly on mathematical reasoning with LLaDA-family models, are substantial.

Critical weaknesses
  1. The methodological novelty relative to the authors’ prior work is insufficiently established. "Reliable Chain-of-Thought via Prefix Consistency", already proposed truncating a reasoning trace, regenerating its continuation, and exploiting the greater reproducibility of correct answers as a reliability signal for voting. PDC applies this core idea to DLMs, replacing autoregressive continuation with suffix denoising in a fixed output window and using an unweighted vote across three prefix lengths. These are meaningful differences, and PDC cites the prior paper. However, the current work does not isolate the benefit of the DLM-specific operation or the multi-prefix voting rule. A direct implementation of the earlier prefix-consistency method on the evaluated DLMs, under a matched budget, is needed to establish the incremental contribution.

  2. Key ablations are missing. The paper reports reproduction rates at individual prefix lengths but does not clearly report the final accuracy obtained by using each length alone. Consequently, it is unclear whether three-length voting is necessary or whether one regeneration accounts for most of the gain. The paper should also compare three regenerations at the same prefix length with one regeneration at each of three lengths, and report how often all three answers differ. In that case, the specified tie rule selects the 10% prefix candidate without a majority agreement.

  3. The evidence supports answer improvement more directly than “verification.” The reported wrong-answer persistence rate measures whether regeneration repeats the same wrong answer; changing to another wrong answer also lowers this rate.

  4. The efficiency claim needs a more complete comparison. The comparison with TiF in the main table involves different inference costs: TiF uses intermediate states from an existing trajectory, whereas PDC runs three additional regenerations. The comparison with four independent generations is useful, but is reported at generation length 128 rather than across the lengths selected for the main results, and its budget is not exactly matched.

Questions for authors

What are the accuracies of the 10%, 50%, and 90% prefix regenerations individually? How often do the three candidates tie or produce no extractable answer?

How does PDC compare, at matched inference cost, with the earlier prefix-consistency voting procedure implemented on the same DLMs?

What fraction of examples change from wrong to correct, correct to wrong, or one wrong answer to another?

Does the 90% retained prefix ever already contain the final answer, and how often?

2 comments

We thank the reviewer for the concrete list of ablations; we ran all of them. Additions are in red in the revised PDF: Appendix I (Tables 13-16, Figure 3), Appendix I.1 (Tables 17-18), Appendix J and Appendix K. Pairs "a / b" below are LLaDA-1.5 / LLaDA-8B.

1. Novelty over PC-WMV (Iwase et al.) and a matched-budget implementation (Table 13).

To address the requested direct comparison, we implemented PC-WMV on the same DLMs and benchmarks at a nominal budget of 2.5T (Table 13). On GSM8K, PC-WMV achieves 61.4 / 61.8, compared with 66.3 / 65.9 for PDC. Across all ten LLaDA settings, PC-WMV stays within 1.1 points of INIT.

The distinction is how regeneration is used. PC-WMV aggregates groups formed from multiple initial samples, whereas PDC regenerates from different prefixes of one initial output and votes only over the regenerated answers. This distinction matters in the tested low-temperature setting: at temperature 0.2, four independent LLaDA generations give the same final answer on 95% of GSM8K problems, and INIT ×2 / ×3 / ×4 stay within 0.6 points of INIT across the ten settings.

These results demonstrate an improvement over the evaluated prior-method implementation. We separately examine whether multi-prefix voting is necessary in the keep-rate ablations below.

2. Each keep rate alone; same rate versus three rates (Appendix I.1, Tables 17-18).

One regeneration at keep rate 0.5 (1.5T) gives 75.3 / 73.5 on GSM8K and 31.1 / 31.9 on MATH-500, above the submitted three-rate vote (66.3 / 65.9 and 26.7 / 27.9); rates 0.1 and 0.9 alone give 61.9 / 60.8 and 64.6 / 63.4 on GSM8K. Three regenerations at 0.5 with different seeds add nothing over one (75.3 / 73.6), so the gain comes from the prefix, not from sampling; the vote over {0.25, 0.5, 0.75} at 2.5T gives 73.4 / 72.9.

The three-rate vote is conservative: the 0.9 candidate repeats an initially wrong answer in 85-88% of cases on GSM8K, so a correction needs both other candidates. In exchange, it never falls more than 0.1 points below INIT in any of the 15 settings, whereas {0.5} loses 0.4-1.0 points on CSQA.

The revision therefore recommends R = {0.5} for derivational tasks and keeps {0.1, 0.5, 0.9} as the default (note at the end of Section 2.2). Appendix I.1 also shows why 0.5 works: where a wrong arithmetic step is detectable in a wrong GSM8K output, the first one lies after the 0.5 cut in 76% / 74% of cases and always inside the 0.9 prefix. In summary, a cost-efficient version of PDC that solely uses regeneration of 0.5 tends to perform as good as the current PDC, while there are some cases in which the robustness of the current PDC helps.

3. Ties and missing answers (Table 14).

All three regenerated answers differ on 8.6% / 6.4% of GSM8K problems, 12.4% / 12.6% of MATH-500 problems and at most 1.3% elsewhere for LLaDA. Ties are broken to prioritize the 0.1 candidate, which is correct on 5.3% / 10.6% of GSM8K ties. No regeneration yields an extractable answer on at most 0.9% of problems for LLaDA.

4. Verification versus answer improvement (Table 14).

We agree that same-wrong persistence alone does not establish verification, and now report transitions for every setting. On GSM8K, 5.8% / 5.0% of problems move wrong -> correct, 0.7% / 0.7% correct -> wrong and 8.0% / 5.7% wrong -> another wrong answer, while 24.9% / 26.8% keep the same wrong answer.

The verification signal is reported separately from accuracy: the number of regenerations agreeing with the initial answer predicts its correctness with AUROC 0.89 / 0.85 on GSM8K, 0.71 on MATH-500 and 0.55-0.61 on CSQA and SQA. When all three agree, the initial answer is correct 90.1% / 87.3% of the time on GSM8K. In summary, PDC not only corrects wrong answers but also reliably predicts the correctness of initial answers through regeneration agreement (AUROC 0.85–0.89 on GSM8K).

5. Efficiency comparison.

Table 2 compares with INIT x4 at the Table 1 lengths; the L=128 comparison is Table 8 (Appendix E). The caption of Table 2 now states the budgets: 2.5T nominal, 2.53-2.56T realized for LLaDA and up to 3.3T when outputs are shorter than the window.

TiF reuses the states of the initial run and costs 1T; we include it as the strongest published DLM-specific method, not as a cost-matched baseline. Table 13 brackets PDC's budget with INIT x2 (2T), INIT x3 (3T) and PC-WMV at exactly 2.5T, and Figure 3 (right) plots accuracy against budget for all keep-rate subsets. In summary, these figures show the advantage of PDC under the same budget.

6. Does the 0.9 prefix already contain the final answer? (last two columns of Table 14)

For LLaDA, the kept 0.9 prefix contains a \boxed{} expression on 0.3-0.6% of GSM8K and MATH-500 problems and about 10% of SVAMP problems. The initial answer string appears in it on 55-56% of GSM8K, 66-67% of MATH-500, 97-98% of SVAMP and 14-15% of CSQA problems. This explains the high same-wrong persistence at keep rate 0.9 on the math tasks. In summary, yes in general.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.