- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether dependency-ready decoding, which reveals tokens in masked diffusion language models only after their unresolved dependencies have been resolved, offers advantages over conventional confidence-based decoding. Empirically, the authors analyze the attention-based groundedness measure used in DOS and find that, under the evaluated settings, first-layer DOS behaves almost identically to left-to-right decoding. They also show that EOS handling substantially affects performance comparisons with confidence-based baselines. Theoretically, the paper introduces a Pre-Commit score, , defined using the KL divergence between predictive distributions before and after revealing the ground-truth value of another masked token. The authors prove that the top-1 prediction at a given position remains unchanged when its prediction margin exceeds . Based on this result, they argue that the potential additional benefits of dependency-ready decoding are limited to positions with small prediction margins.
Critical strengths
-
Re-examining not only the performance of existing methods but also their underlying mechanisms is an important research direction. The finding that DOS(1) without the EOS fix achieves exactly the same reported accuracy as left-to-right decoding across all evaluated tasks for both models raises questions about whether the performance improvements of DOS can genuinely be attributed to dependency modeling.
-
The proposition itself is sound and verifiable. The stability condition for a single token revelation, formulated in terms of the prediction margin and KL divergence, follows from Pinsker's inequality and is mathematically valid within this limited scope.
Critical weaknesses
-
Overgeneralization of the Theoretical Result The proposition provides a sufficient condition based on the relative magnitude of the prediction margin and unresolved sensitivity, rather than the margin alone. Even with a large margin, the prediction may change if the sensitivity is sufficiently high. Therefore, the conclusion that the additional benefits of dependency-ready decoding are limited to small-margin positions is not theoretically justified.
-
Missing Connection Between Prediction Stability and Decoding Order The proposition only guarantees that the top-1 prediction at a given position remains unchanged. It does not establish that max-margin decoding selects positions similar to those chosen by an oracle dependency-ready policy. Therefore, the claim that max-margin decoding already captures dependency readiness lacks a sufficient logical foundation.
-
Insufficient Justification of Pre-Commit as an Ideal Readiness Measure Pre-Commit measures changes in predictions following the revelation of a single token, but it does not account for the joint effects of revealing multiple tokens or guarantee prediction correctness. In particular, Pre-Commit can be small even when an incorrect prediction remains unchanged after additional information is revealed. Thus, the justification for treating Pre-Commit as an ideal criterion for dependency readiness is insufficient.
Questions for authors
See the weaknesses.
3 comments
Thank you for the detailed response and additional experiments. Many of my concerns have been substantially addressed, although the theoretical justification for the central claims remains limited. Nevertheless, I appreciate the improvements and have decided to raise my score to 2.
Response to Reviewer V7bD
We sincerely appreciate your thoughtful consideration and raising your score. In the revision, we have taken more care to avoid overclaiming, and perhaps our revisions will be easier to find with the newly uploaded version marking many of them in red. If you could point out with a line number which specific central claim you believe is still insufficiently justified, we will try to address the problem.
Response to Reviewer V7bD
We thank the reviewer for the careful review.
W1. The relative margin condition.
We have worded our conclusions more carefully to emphasize the role of the Pre-Commit score and not just the margin in the criterion, and we added experiments to test how frequently the condition was satisfied at various ranges for the margin. Figure 6b in newly added Section 6.3 gives the full picture. The share of maximum-margin commits that a single reveal can flip falls from 45.6% at margin 0.3–0.5 to 36.7% at 0.5–0.7, 25.9% at 0.7–0.9 and 16.8% at 0.9–1.0. A larger margin makes a flip less likely but never rules it out, so the revision keeps only the condition relative to 2μ.
W2. Maximum-margin decoding and stable positions.
We have removed the suggestion of an "oracle dependency-ready policy" and instead focused on predicting positions whose margin is sufficient to guarantee stability with respect to single token revelations. Our new empirical analysis of 9×9 Sudoku in Section 6.3 shows that predicting such stable tokens (which is more likely when the criterion is satisfied) leads to higher accuracy, and also that maximum margin decoding tends to choose positions satisfying the criterion more often than DOS.
W3. Joint effects, Pre-Commit and correctness.
Joint effects can be addressed by the same argument, and we add this comment after the Proposition in Section 6.2. A set of k revealed tokens defines a joint score μtk(v), and larger k gives a stricter criterion on the margin.
Although stability with respect to single token revelations does not guarantee correctness, stable (including those satisfying the Pre-commit criterion) commits are far more often correct on 9×9 Sudoku: 56–77%, against 8–21% of unstable ones (c.f. newly added Section 6.3), and unstable commits account for more than half of each decoder's errors.