- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper explains why parallel decoding in masked generation models can be hard without looking at already generated values. The authors construct a toy four-symbol target in which every pair of positions is independent. A decoder that uses generated values can sample this target exactly in three rounds, but a decoder that ignores them needs at least omega(L/log L) rounds to keep the error small. However, for uniform sampling without replacement, generated values do not help.
Critical strengths
- Fig. 1 is easy to interpret, and the idea is clear.
- The main idea, that a fixed schedule cannot use information revealed during generation, is intuitive, and the construction combines known tools such as secret sharing, orthogonal arrays, and hidden hypermatchings.
Critical weaknesses
Weaknesses are mainly about visualization, clear explanation, and practicality. Details are explained in the questions.
Questions for authors
- Fig. 2’s format is hard to understand. (c) and (f) use the same caption “four scales” but it is shown differently in each plot: in (c) the four lines are the four sizes, while in (f) the sizes are on the x-axis and the four lines are leakage levels instead. “Naïve split”, “Naïve fail”, “Naïve witnesses” are not clearly distinguished, and “Naïve fail”, “Naïve witnesses” seem to refer to the same 4,296 cases. These terms are confusing.
- Even if this “ambiguity in next token prediction” happens in this toy example, is there a guarantee that real data has similar scenario? Lack of practical experiment (no trained model or real data is used) is the key weakness of the paper.
- The three-round decoder does not just look at generated values. It uses them to compute the hidden grouping with the full route table. Is the generated values itself necessary, or knowing the hidden structure is necessary?
- Is adaptive decoder explanation in Appendix F (e.g., confidence-based) actually faster than value-oblivious decoders on this example?
- Constants in page 6 and Lemma A.1,2 (1/9, 1/18, five ties) come from loose bounds, while the experiments show losses of about 0.5. Can the bound be tightened? Also, the theorem certifies error floor of 5.4%, and 5% threshold is arbitrarily chosen. What about larger tolerance decoder? At least the calculations and numbers should be tightened and generalized. The number that is guaranteed from the theorem is not strong enough.
- What if we do remasking such as ReMDM (Wang et al., 2025) or PRISM (Kim et al., 2026) or LLaDA 2.1 (Bie et al., 2026)? If the decoder can circumvent by remasking, this might not be a problem.
- The novelty compared with known adaptive vs. non-adaptive separations is unclear. What is technically new beyond applying this idea to masked generation?
(Wang et al., 2025) G. Wang, Y. Schiff, S. S. Sahoo, and V. Kuleshov. Remasking Discrete Diffusion Models with Inference-Time Scaling. NeurIPS, 2025. arXiv:2503.00307. (Kim et al., 2026) J. Kim, S. Kim, T. Lee, D. Z. Pan, H. Kim, S. Kakade, and S. Chen. Fine-Tuning Masked Diffusion for Provable Self-Correction. ICML, 2026. arXiv:2510.01384. (Bie et al., 2026) T. Bie, et al. LLaDA2.1: Speeding Up Text Diffusion via Token Editing. arXiv:2602.08676, 2026.