- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper develops an adjoint-equation framework for analyzing the convergence of score-based discrete diffusion models. By studying errors in observable expectations, it derives bounds for integral probability metrics in terms of initialization and score estimation errors. For masked and uniform transitions, the bounds have coefficients without explicit vocabulary-size dependence, using score–marginal cancellation, exit-rate estimates, and coupling arguments. The analysis also covers early stopping and provides expected step-complexity guarantees for uniformization under additional assumptions. The contribution is theoretical, supported by mathematical proofs rather than empirical experiments.
Critical strengths
-
The observable-based formulation provides a unified way to separate initialization and score estimation errors and connect them to different IPMs (Lemma 3.2; Theorem 3.1). This makes the analysis useful beyond a single distributional metric.
-
The score–marginal cancellation exposes an exit-rate coefficient rather than an entrance-rate coefficient, avoiding explicit vocabulary-size growth for masked transitions (Remark 3.2; Corollary E.7). Its interpretation as the expected number of forward jumps provides a useful explanation of how the dynamics affect the error bound (Remark E.8).
Critical weaknesses
-
Corollary 3.4 omits an essential condition on the time grid. In Eq. (91), the factor , where , is absorbed into a constant without requiring to remain bounded independently of the target accuracy. For a single-token masked model with and exact scores, using one interval gives . With and , this scales as , rather than the claimed logarithmic complexity. Thus, the stated sampling guarantee requires an explicit restriction on the grid.
-
The adjoint argument needs a rigorous treatment of the singular endpoint at . Assumption 1 concerns the forward generator and does not ensure that Eq. (13) admits a bounded continuous solution for arbitrary initial observables. For a single-token masked model with data concentrated on a non-mask token , , and exact scores, the reverse unmasking rate is . Choosing and yields , which has no bounded continuous solution satisfying . This leaves a gap in the justification of Lemma E.1 and the subsequent error identity on . A truncated-interval argument with a justified endpoint limit is needed; the example does not itself disprove the final distributional bound.
Questions for authors
-
Does Corollary 3.4 require a time grid with bounded independently of ? Please clarify how the factor in Eq. (91) is controlled and specify a grid that achieves the stated complexity. How does the statement account for the single-interval example described above?
-
How is the adjoint equation in Eq. (13) defined at when reverse rates are singular, as in the single-token masked example above? Can the authors justify the error identity by working on and passing to the limit , or specify the additional assumptions or boundary interpretation needed? Please clarify whether this changes the scope of Theorem 3.1.
3 comments
Follow-up on our responses
Dear Reviewer nsG6,
As the discussion period ends on November 18, we would be grateful to know whether our responses have addressed your concerns. We are happy to clarify anything further.
Best regards,
Authors
Thank you for the clarifications. The bounded-ratio grid condition addresses my concern about the uniformization complexity. The explanation of the early-stopping analysis also substantially alleviates my concern about the singular endpoint. Please make the assumptions and scope of the results without early stopping explicit in the revised manuscript. In light of these responses, I am updating my rating to 3 (weak acceptance).
Rebuttal
Q1. Time-grid condition in Corollary 3.4. We thank the reviewer. Corollary 3.4 indeed requires a grid whose ratio R=maxktk+1/tk is bounded independently of δ; we have stated this assumption explicitly in the updated manuscript. A simple example to achieve this is tk+1=2tk, so (R−1)/logR=1/log2 and the stated complexity holds.
Regular grid assumptions are standard in the literature. Theorem 3 of [1] and Theorem 4.9 of [2] similarly impose a regular grid condition in their step-complexity analyses of uniformization, for the constant schedule β=1; we consider the more general variable schedule.
This assumption is not restrictive. Refining the grid will not increase the expected step complexity in uniformization. This is because grid points require no network evaluations, and only Poisson events do. Since the Poisson rate only needs to dominate the transition rate within each grid interval, a finer grid allows for a tighter Poisson rate on each interval, and hence fewer expected steps.
The single time interval example has R=T/δ and is excluded by this condition. It is also impractical. This is because a single Poisson rate must dominate the transition rate over the entire horizon [δ,T], resulting in a loose Poisson rate and inefficient simulation.
Q2 Singular endpoint at t=0. We thank the reviewer for pointing this out. This case is already covered in the manuscript, and we will modify the manuscript to make it clearer.
To keep the presentation clear without introducing too many concepts at once, Sections 2, 3.1, and 3.2 first treat an idealized setting without early stopping, in which the data is fully supported at t=0 and the reverse rates are bounded.
Section 3.3 (Eq. (20)) then presents the results with early stopping, which is essential when the data is not fully supported, as in masked diffusion, where the data has no mass on the all-masked sequence. In this setting, the reverse rates are bounded on [δ,T]. The derivation, including Lemma E.1, follows Section 3.1 verbatim with [0,T] replaced by [δ,T]. And the bound has an extra early-stopping term.
We emphasize that the early-stopping result appears later in Section 3 only for clarity of presentation. Our results already apply to the case the reviewer raised, including the singular endpoint in their example.
Q3 "The contribution is theoretical, supported by mathematical proofs rather than empirical experiments." We thank the reviewer, Please refer to the response to Q3 of reviewer ifzu.
[1] Liang, Y., Huang, R., Lai, L., Shroff, N., & Liang, Y. (2026). Absorb and converge: Provable convergence guarantee for absorbing discrete diffusion models. Advances in Neural Information Processing Systems (NeurIPS 2025), 38, 20283-20318.
[2] Ren, Y., Chen, H., Rotskoff, G., & Ying, L. (2025, May). How discrete and continuous diffusion meet: Comprehensive analysis of discrete diffusion models via a stochastic integral framework. In International Conference on Learning Representations (ICLR 2025), 42904-42941.