acceptodds
Withdrawn from ICLR 2027

REVISE WHAT MATTERS: ATTENTION-PRIORITIZED VERIFICATION FOR DIFFUSION LANGUAGE MODELS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion language models enable parallel token generation, but existing revocable decoders often treat uncertainty as sufficient evidence for revision, causing many suspicious predictions to be remasked even when revision provides little benefit. We argue that effective revision should consider not only whether a token is unreliable, but also whether correcting it is likely to matter to the evolving decoding context. We propose CAPER (Context-Aware Probation for Efficient Revision), a training-free revocable decoding framework that introduces a one-step probation state between token commitment and remasking. CAPER combines assignment uncertainty with incoming attention from already decoded and newly drafted positions to prioritize contextually consequential candidates for verification. During probation, priority-weighted drafting limits the influence of unresolved predictions on concurrent generation, while verification determines whether their current assignments should be retained or revoked. Drafting and verification are integrated into the same decoding step, allowing CAPER to concentrate revision on actionable predictions without sacrificing parallel generation. Experiments on LLaDA-8B-Base and Dream-Base-7B across code generation and mathematical reasoning demonstrate a consistently stronger quality–efficiency trade-off than existing irreversible and revocable decoding methods.

Primary area:
foundation or frontier models, including LLMs
Keywords:
DLM remask

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7wfXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper first presents several key insights: 1. in 98% cases the tokens that got remasked will be decoded in the immediate unmask in the next step, and only 29% tokens become different from the tokens before remasking; 2. the changed tokens receive more attention from other tokens than the preserved tokens. Based on the insights, the authors argue that remasking should only happen when it will change a token, which is important for the generation process. They propose a training-free decoding framework that introduces a one-step probation state.

Critical strengths
  1. The method is well motivated by the discovered remasking behavior. And the ablation experiments are thorough.
  2. The method outperforms baselines in many settings.
Critical weaknesses
  1. Some components like priority-weighted drafting do not seem to be very well-motivated. This may make the method not generalizable. See question 1.
Questions for authors
  1. The priority-weighted drafting does not make sense to me, equation 9 in particular. It extrapolates the mask embedding from the aggregated prediction, and training in general does not seem to support an input embedding like this. I would consider raising the score if the authors can provide a sound theoretical justification.

  2. In Figure 4, the score does not improve monotonically with probation budget K. Can you provide an explanation?

Reviewer_kkjWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CAPER, a training-free decoder that verifies selected committed tokens before remasking them. Tokens enter a one-step probation state based on uncertainty and incoming attention. Drafting and verification share one forward pass, using cached key/value states and a diagonal correction. Experiments on LLaDA-8B-Base and Dream-Base-7B cover reasoning and code generation at lengths 256 and 512.

Critical strengths
  1. The problem is well motivated. Many remasked tokens quickly return to their previous values, so separating suspicion from immediate remasking is a useful idea. The probation mechanism provides a concrete way to reduce this wasted work while retaining the ability to revise earlier predictions.

  2. Good reported performance across the tested settings. On Dream GSM8K at length 512, CAPER matches Saber's 76.80% accuracy while reducing forward evaluations from 283 to 48. The ablations also support the roles of guided drafting, attention-based selection, and verification.

Critical weaknesses
  1. The efficiency comparison would be stronger with a matched adaptive irreversible baseline. The current irreversible baseline uses one step per generated position, while CAPER uses confidence-based parallel drafting. The results do not fully separate the gain from adaptive drafting from the gain of the proposed revision mechanism.

  2. The verification approximation needs more validation. The method retains cached representations computed with the original probation assignments and corrects only the checked token's direct self-entry. Other cached states can still contain information about that assignment. A comparison with clean leave-one-out verification would clarify the effect of indirect self-conditioning and stale context.

Questions for authors

N/A

Reviewer_nidCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes CAPER, a training-free revocable decoding method for diffusion LLMs. It starts from the observation that most remasked tokens are restored unchanged at the next step, while tokens that do change receive higher incoming attention. Instead of remasking a low-confidence token right away, CAPER puts the top-K tokens, ranked by uncertainty x (1 + attention from decoded tokens) x (1 + attention from drafted tokens), on a one-step “probation”. At the next step these tokens are fed as [MASK] for verification. Other positions still attend to their cached K/V from the previous step, and a diagonal correction stops each token from seeing itself. The token is kept if it is re-predicted with confidence above threshold and remasked otherwise. Drafting and verification share one forward pass, and drafts are pushed away from the probation tokens’ embeddings.

Critical strengths

S1. The motivation is backed by clear empirical analysis. Fig. 1 shows that 98% of token-preserving remasks recover at the next step, and Fig. 3c links incoming attention to effective remasks.

S2. The method is training-free and runs drafting and verification in a single forward pass. It cuts remask counts by orders of magnitude while raising the effective remask rate, across different methods and benchmarks.

Critical weaknesses

W1. My major concern is the overlap with COVER [1] (ICML 2026), which is not cited. COVER already proposes:

  • the same flip-flop motivation (remask, then restore the same token);

  • the same single-pass verification -- seeds are masked in the input, other queries use the seeds’ cached K/V from the previous step, and a diagonal correction with row renormalization prevents self-leakage. Sec. 4.2 even uses COVER’s title phrase “context-preserving verification”;

  • the same seed score form, u x (1 + attention terms), computed from last-layer, head-averaged attention; KEEP/REMASK resolution;

  • the same setup: B=15, tau=0.8, lengths 256/512, the same four benchmarks, WINO/Saber baselines and the effective-revision-ratio metric.

The remaining differences are small:

  • the attention terms in the score are changed;

  • K is fixed at 2 instead of COVER’s adaptive budget;

  • REPLACE is dropped;

  • guided drafting (Eq. 9) is added. This is close in spirit to anchor-guided injection into masked positions in ASRD [2].

The flip-flop observation itself is also reported in [3]. The authors should cite and compare against [1], and present the contribution as these deltas, with ablations against COVER.

W2. The speedup is measured against one-token-per-step decoding, so most of it likely comes from threshold parallel decoding rather than revision. Threshold decoding without revision (Fast-dLLM [4]) is the necessary baseline. [3] shows that WINO’s gain over Fast-dLLM is marginal at standard block lengths. [3] also shows that confidence read at an unmasked position, which is how u is computed here, does not help remasking.

W3. The efficiency numbers need explanation. With B=15 and block length 64, each block needs at least 5 steps, so going from 256 to 512 tokens should add at least 20 NFE. Yet CAPER’s NFE stays flat or drops: Dream GSM8K goes 66 ---> 48, Dream HumanEval 70 ---> 54, and LLaDA GSM8K 71 --> 71. WINO and Saber both increase. Is early stopping at EOS used for CAPER but not for the baseline? Also, Dream GSM8K at 512 reports a 12.47x wall-clock speedup, above its NFE ratio of 10.7x, even though CAPER needs attention maps and K/V caching at every step.

W4. Several technical choices are not justified:

  • Verification only removes the self-entry. Neighbors attend to the cached K/V of the token at every layer, so from layer 2 onward the token can still see its own value indirectly, which biases the check toward KEEP. There is no comparison with an oracle leave-one-out check, as done in [3].

  • Eq. 9 has no scale factor. Since all masked positions share the same [MASK] embedding, it reduces to one out-of-distribution shift, 2e_MASK − e_P, and there is no sensitivity study.

W5. Results come from a single greedy run, and the gains are small; for example, +2.43 on HumanEval is 4 problems. Fig. 1b–c pools about 1k token-changing events across 2 models and 4 tasks, and is drawn as interpolated histograms.

W6. The Ethics and Reproducibility sections are left as the unedited template. The paper writing seems to be rushed. Very few papers are discussed in related work and in Introduction.


References

  1. Xiang, Yanzheng, et al. “Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding.” International Conference on Machine Learning. 2026. (arXiv:2602.06161)

  2. Yao, Yizhen, et al. “Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens.” arXiv preprint arXiv:2606.16847 (2026).

  3. Frković, Stipe, et al. “Re-evaluating Confidence Remasking in Masked Diffusion Language Models.” arXiv preprint arXiv:2606.12232 (2026).

  4. Wu, Chengyue, et al. “Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding.” International Conference on Learning Representations. 2026.

Questions for authors

please check weaknesses for questions.

2 comments

\section*{Response to Reviewer nidC}

Thank you for the detailed review. We added a direct COVER comparison, an Exact LOO evaluation, and an analysis of Eq.~9.

\paragraph{W1. Attribution and contribution.} We now cite COVER and Frkovi'{c} et al.\ and include COVER in Table~1. We credit COVER with the flip-flop observation, cached-KV verification with diagonal correction, and the shared drafting and verification pass.

CAPER combines uncertainty with incoming attention from decoded and newly drafted positions to prioritize verification. COVER rewards influence on masked positions and penalizes outgoing attention to new drafts as a proxy for cache instability. CAPER remasks when verification changes the top-1 prediction or when its confidence falls below the threshold.

Inspired by SFG, CAPER shifts masked embeddings away from a priority-weighted aggregate of unresolved assignments. ASRD guides them toward an unweighted centroid of stable anchors using entropy-dependent strengths. CAPER scores higher than COVER in 14 of 16 settings and is at least as fast in all 16. Table~2 evaluates the components within CAPER.

\paragraph{W2. Parallel decoding and confidence.} The single-token decoder supplies the speedup denominator. On Dream GSM8K at length 512, CAPER is 1.30×1.30\times faster than WINO and 1.97×1.97\times faster than COVER, with higher accuracy. Replacing verification with immediate remasking preserves parallel drafting, lowers all four scores, and reduces normalized performance to 0.3330.333--0.383×0.383\times. This ablation measures the benefit of verification within revocable decoding. Isolating revision requires a matched irreversible parallel decoder.

Frkovi'{c} et al.\ study unmasked confidence as a direct remasking criterion. CAPER combines it with attention to select candidates and uses masked verification for the final decision.

\paragraph{W3. EOS, NFE, and runtime.} Revocable methods use EOS-based early termination. The reference uses a fixed budget of 256 or 512 steps. The five-step bound applies to fully decoding one 64-token block under the 15-token-per-step budget. A longer allocation can change how many blocks are actually decoded. For example, WINO on Dream HumanEval goes from 75.27 to 74.77 forwards. The aggregate results leave the cause unresolved.

The 12.47×12.47\times runtime speedup and 512/48≈10.67512/48\approx10.67 forward-count ratio imply that the reference uses about 1.17×1.17\times as much GPU time per counted forward. Identifying the source of this difference requires component-level profiling.

\paragraph{W4. Verification and guidance.} Diagonal correction can leave indirect self-conditioning paths. In Appendix Sec.~3, Exact LOO masks only the target and recomputes the full model without cached KV states. With guidance disabled for both verifiers, top-1 and decision agreement across 151,281 events are 99.37% and 97.74%. Exact LOO reverses 1.34% of KEEP and 24.87% of REMASK decisions. In 89.22% of disagreements, top-1 stays fixed while confidence crosses the threshold. Using Exact LOO during generation increases mean accuracy by 0.38 points, NFE by 44.11%, and measured runtime by 36.56%, including diagnostic overhead. Fidelity with guidance enabled remains untested.

Appendix Secs.~1--2 show that under linear query projection and fixed keys and values, the shared shift reduces relative attention to keys with greater excess alignment to the probation aggregate. The shift depends on the selected assignments and priorities. Reversing guidance at −0.5-0.5 lowers scores by 19.00--72.10 percentage points versus zero guidance. Coefficient-1 results come from a separate setting, leaving the sweep optimum unresolved.

\paragraph{W5. Reliability.} Repeating greedy decoding does not quantify uncertainty across test problems. CAPER matches or exceeds the irreversible reference in all 16 settings at 2.072.07--12.47×12.47\times speedups. Figure1(b,c) describes pooled, binned events. Separately, removing contextual attention in Table2 lowers all four scores and reduces the mean effective remask rate from 26.37% to 12.76%.

\paragraph{W6. Presentation and reproducibility.} We clarified attribution and documented the experimental settings and verification protocol. The Ethics and Reproducibility template passages still require correction.

\section*{Response to Reviewer nidC}

Thank you for the detailed review. We added a direct COVER comparison, an Exact LOO evaluation, and an analysis of Eq.~9.

\paragraph{W1. Relationship to Saber, COVER, and ASRD.} Saber and COVER both add token positions to test whether a decoded assignment should be remasked. COVER's key insight is the remask flip-flop phenomenon. By avoiding the first remask category identified in our paper, COVER improves the effective remask rate. We credit COVER with this observation and its shared drafting and cached-KV verification pass with diagonal correction. We now cite COVER and include it in Table~1.

CAPER combines uncertainty with incoming attention from decoded and newly drafted positions to identify revision candidates in both remask categories. COVER rewards influence on masked positions and penalizes outgoing attention to new drafts. CAPER remasks when verification changes the top-1 token or when the same top-1 token falls below the confidence threshold. CAPER achieves the highest effective remask rate in both categories among evaluated methods. Removing contextual attention lowers all four scores and reduces the mean effective remask rate from 26.37% to 12.76% (Table~2).

Inspired by SFG, CAPER shifts masked embeddings away from a priority-weighted aggregate of unresolved assignments. ASRD guides them toward an unweighted centroid of stable anchors using entropy-dependent strengths. CAPER scores higher than COVER in 14 of 16 settings and is at least as fast in all 16.

\paragraph{W2. Parallel decoding and confidence.} The fixed-step single-token decoder is the speedup reference. On Dream GSM8K at length 512, CAPER is 1.30×1.30\times faster than WINO and 1.97×1.97\times faster than COVER, with higher accuracy. Replacing verification with immediate remasking retains parallel drafting, lowers all four scores, and reduces normalized performance to 0.3330.333--0.383×0.383\times. The matched irreversible parallel comparison remains open.

Frkovi'{c} et al.\ study unmasked confidence as a direct remasking criterion. CAPER combines it with attention for candidate selection and makes the final decision using masked verification.

\paragraph{W3. EOS, NFE, and runtime.} Revocable methods stop at EOS. The reference uses a fixed budget of 256 or 512 steps. The five-step bound applies to one fully decoded 64-token block at up to 15 tokens per step. A longer allocation can change how many blocks are completed. For example, WINO on Dream HumanEval drops from 75.27 to 74.77 forwards. The trajectory changes behind this decrease remain unresolved.

The 12.47×12.47\times runtime speedup and 512/48≈10.67512/48\approx10.67 NFE ratio imply that the reference uses about 1.17×1.17\times as much GPU time per counted forward. Attribution requires component-level profiling.

\paragraph{W4. Verification and guidance.} Diagonal correction can leave indirect self-conditioning paths. Exact LOO masks only the target and recomputes the model without cached KV states. With guidance disabled, top-1 and decision agreement across 151,281 events are 99.37% and 97.74%. Exact LOO reverses 1.34% of KEEP and 24.87% of REMASK decisions. In 89.22% of disagreements, top-1 stays fixed while confidence crosses the threshold. Exact LOO increases mean accuracy by 0.38 percentage points, NFE by 44.11%, and measured runtime by 36.56%, including diagnostic overhead. Agreement with guidance enabled remains untested (Appendix Sec.~3).

With linear query projection and fixed keys and values, Eq.~9 reduces relative attention to keys with greater excess alignment to the probation aggregate. The shared shift depends on the selected assignments and priorities. Reversing guidance at −0.5-0.5 lowers scores by 19.00--72.10 points versus zero guidance. The coefficient-1 reference comes from a separate setting, leaving the sweep optimum unresolved (Appendix Secs.~1--2).

\paragraph{W5. Reliability.} Uncertainty across test problems requires problem-level analysis. CAPER matches or exceeds the irreversible reference in all 16 settings with 2.072.07--12.47×12.47\times speedups. Figure1(b,c) describes pooled, binned events. Table2 tests contextual selection at the task level.

\paragraph{W6. Presentation and reproducibility.} We clarified attribution and documented the evaluation settings and verification protocol. The Ethics and Reproducibility placeholders still require correction.

Comments on the paper

1 comment
~Yanzheng_Xiang2OpenReview ↗

This submission presents our ICML 2026 paper COVER as its own work

Dear Area Chair, Reviewers and Authors,

I am the first author of COVER, "Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding" (ICML 2026; arXiv:2602.06161, February 2026; code: github.com/xyzCS/COVER). We have no connection to this submission.

This goes beyond a missing citation. The core components of this submission come from COVER and are presented as new work:

  1. Method: The core verification step (Section 4.2) is COVER's method, presented as the paper's own contribution and titled with COVER's title phrase, "context-preserving verification". The checked positions are input as [MASK], other queries use their cached K/V from the previous step, a diagonal correction prevents self-leakage, and drafting and verification share one forward pass. All of this is in COVER.
  2. Figure: Figure 2 is almost a copy of COVER's Figure 2. It has the same layout, colour scheme, token legend and component labels ("Selected from step t-1", "Cached K,V", "Input as [MASK]", "Attention Diagonal Correction", "Keep / ReMask", "Uncertainty u → Score → Top-K"). The main visible change is that "seed" is renamed "probation".
  3. Code: The supplementary code is built on COVER's released code and cannot run without it. caper/run.py imports COVER's model files by their repository paths (llada_base.code.modeling_llada_kv_cover, dream_instruct.code.lm_eval.models.cover.modeling_dream), and caper/decoder.py runs verification through COVER's _seed_kv_* hooks. The paper calls them "probation tokens", but the code still uses COVER's "seed" names.
  4. Setup: The same models, block length 64, threshold 0.8, draft budget 15, and the same four benchmarks and baselines.

Even so, the submitted version does not cite COVER anywhere.

On the rebuttal. The authors now credit COVER with "the flip-flop observation, cached-KV verification with diagonal correction, and the shared drafting and verification pass". This confirms that the core of the method is COVER's, yet the submitted version presented it as the paper's own contribution. Adding a citation only after a reviewer pointed this out does not change what was submitted. The rebuttal also does not address the copied Figure 2 or the code built on COVER's repository. Once COVER is credited, what remains is a different candidate score, a fixed probation budget and a guided drafting step that the rebuttal itself describes as inspired by SFG.

We have reported this to the Program Chairs with a detailed comparison. We ask the Area Chair to investigate, and the authors to explain why COVER’s figure and code were used without any credit.

Yanzheng Xiang

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.