- Rating
- 3
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper compares repeated local remasking of a completed diffusion-language-model response with independent sampling. Across four models and three main reasoning/code benchmarks, repair produces fewer distinct answers and substantially lower oracle pass@, despite sometimes improving accuracy. The contribution is an empirical diagnosis using answer agreement, coverage, correction or regression rates, remasking interventions, and budget comparisons. The results favor spending inference compute on independent trajectories before extending a single repair chain.
Critical strengths
-
The empirical gap is substantial and practically relevant. At similar initial accuracy, LLaDA-8B reaches 82.8% oracle pass@16 with independent sampling versus 29.0% with repair on Countdown (Table 2). Approximately compute-matched comparisons also favor independent sampling, even when repair can select from more candidate states. This provides useful evidence for allocating inference compute.
-
The paper checks several plausible alternative explanations. Chain-level selection, semantic canonicalization, parameter sweeps, and guided-remasking interventions strengthen the observation beyond a simple accuracy comparison. Reporting exceptions, including Dream's Countdown gains and favorable creative-writing results, also helps establish the finding's scope.
Critical weaknesses
-
The central dependence explanation remains unresolved. Partial repair uses joint regeneration, whereas independent sampling uses 32-token blockwise decoding. In Table 18, changing the full-remask regeneration schedule alone raises Countdown pass@16 from 46.8% to 80.0%, close to the 83.0% independent baseline. This is a large effect without retained-state dependence. Matching the first candidate's accuracy does not match later candidates' distributions, and also reflects marginal answer concentration. The authors acknowledge these limitations, but they leave unclear how much dependence contributes to the measured gap.
-
The contribution beyond closely related work needs stronger support. Frković et al. [1] already examine limited remasking benefits, decoding-setting sensitivity, and diversity loss. This work is absent from the discussion. The present focus on successive states within one repair chain is distinguishable, but the experiments use a generic post-hoc repair operator without directly evaluating an established decoder such as WINO [2] or CORE [3]. The resulting advance over existing remasking analyses is not yet clear.
References
[1] Frkovic, Stipe, et al. "Re-evaluating Confidence Remasking in Masked Diffusion Language Models." arXiv preprint arXiv:2606.12232 (2026).
[2] Hong, Feng, et al. "Wide-in, narrow-out: Revokable decoding for efficient and effective dllms." International Conference on Learning Representations. Vol. 2026. 2026.
[3] Zhai, Kevin, et al. "Core: Context-robust remasking for diffusion language models." arXiv preprint arXiv:2602.04096 (2026).
Questions for authors
-
Evidence needed for the central explanation: Does the partial-remasking coverage gap persist when repair uses the same blockwise regeneration kernel and comparable denoising allocation? A comparison between within-chain candidate sets and candidates drawn across independently initialized repair chains at matching round indices would help separate serial dependence from marginal concentration.
-
Contribution and scope clarification: Which findings go beyond Frković et al. [1], and do those findings hold for an established remasking method, such as WINO [2] or CORE [3], beyond the post-hoc operator evaluated here?
-
Implementation clarification: Does Dream's confidence recomputation apply the one-position logit shift used in its official sampler? Table 9 reports answer confidences of only 0.008–0.014. Please explain these values and confirm the alignment in both the repair operator and the confidence audit. These values warrant clarification but do not, by themselves, establish an implementation error.
References
[1] Frkovic, Stipe, et al. "Re-evaluating Confidence Remasking in Masked Diffusion Language Models." arXiv preprint arXiv:2606.12232 (2026).
[2] Hong, Feng, et al. "Wide-in, narrow-out: Revokable decoding for efficient and effective dllms." International Conference on Learning Representations. Vol. 2026. 2026.
[3] Zhai, Kevin, et al. "Core: Context-robust remasking for diffusion language models." arXiv preprint arXiv:2602.04096 (2026).
4 comments
Response to Reviewer LP6F (2/2)
Comment/Question:
- Evidence needed for the central explanation: Does the partial-remasking coverage gap persist when repair uses the same blockwise regeneration kernel ...
Answer: Yes to both parts; see our answer to Weakness 1 above, with the tables in GR-1 and GR-2. Under the same kernel and denoising allocation the gap persists, and pools compared at matching round index show that across-chain pools behave like independent sampling while within-chain pools do not.
Comment/Question:
- Contribution and scope clarification: Which findings go beyond Frković et al. [1], and do those findings hold for an established remasking method ...
Answer: See our answer to Weakness 2 above and GR-3. In short, the chain-level separation through the matched kernel and the within- versus across-chain pools goes beyond Frković et al. Among established methods, the released decoders that produce independent trajectories do not concentrate, while repair chains using CORE- or WINO-inspired selectors do.
Comment/Question:
- Implementation clarification: Does Dream's confidence recomputation apply the one-position logit shift used in its official sampler? ...
Answer: The repair operator applies the shift; the separate audit did not. Dream predicts token i from position i−1, and our repair operator shifts the logits accordingly before the softmax, so all Dream repair results are unaffected. The audit's omission explains the anomalous confidence values. With the shift, answer confidence is about 0.95 on both tasks, and the exposure–agreement correlation is unchanged. GR-4 gives the corrected values, and Tables 10–11 and the text are corrected. Thank you for raising this.
Thank you for providing additional evidence and addressing my concerns about the paper’s novelty. I have no further questions. I will maintain my rating and increase my confidence score from 3 to 4.
Dear Reviewer LP6F,
Thank you for your follow-up and for engaging so carefully with our response. Your suggestions, in particular the within-chain versus across-chain comparison, the blockwise kernel control and the Dream alignment check, made the paper clearer and more rigorous, and the revision now reflects them.
We are grateful for your support and for the increased confidence.
Best regards,
The Authors
Response to Reviewer LP6F (1/2)
We thank the reviewer for the careful review. The shared evidence is in the General Response, GR-1 to GR-4.
Comment/Question:
Answer: We ran both controls the reviewer proposes, and they separate the explanations.
Kernel control (GR-1). With the independent sampler's blockwise kernel and a matched denoising budget, partial repair at α=0.2 still falls 17–54 pp below independent sampling on all three tasks, with every interval excluding zero; Dream-7B is similar. The schedule effect the reviewer cites from Table 18, now Table 19, is real at α=1, but with 80% retained the same swap moves coverage by only 0.5 pp.
Serial dependence versus marginal concentration (GR-2). Pooling the final states of eight independently initialized chains gives coverage with no detectable difference from independent sampling on all three tasks, and nearly the same agreement. Pooling the states within one chain loses 18–50 pp. The marginal distribution of repaired states is therefore not concentrated; the gap comes from dependence between successive states of the same chain. This is the comparison the reviewer suggested, and it is in Appendix G.2.
The revised Section 5.2 states the dependence claim at this level, together with the limitation that retained context, bidirectional attention and schedule are not fully decomposed.
Comment/Question:
Answer: Thank you for the reference; the revision now cites and discusses Frković et al. They compare decoders that remask during generation, namely WINO, Saber and COVER, against confidence-based unmasking alone. They find that remasking matches the baseline at short block lengths and helps only at longer blocks, that WINO loses diversity under stochastic decoding, and that the dLLM rather than the remasking rule is the bottleneck.
Our question is different: whether the successive states of a post-completion repair chain form a useful candidate pool for test-time scaling. Three findings go beyond their work.
The third point places the effect in sequentially re-editing one retained chain, not in remasking as such. This agrees with Frković et al.'s observation that remasking is not the bottleneck, and it answers a question their decoder-level comparison does not ask.