- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Variational Stackelberg Discrete Diffusion (VSDD), which updates the forward corruption process on-policy with learnable embedding matrices, instead of relying on a uniform forward process. The authors formulate training as a leader–follower game, where the leader optimizes the transition matrix () based on the follower's improvement after a single gradient update, evaluated under a fixed reference corruption process. The follower minimizes the standard D3PM loss. Experiments across comparatively small domains (molecular, text, and playlist generation) demonstrate overall improvements, while the gains in the text domain lag behind masked diffusion models.
Critical strengths
-
The proposed game-theoretic formulation is novel and seems to mitigate the issues associated with previous joint learning of the forward process and denoising model. This is a meaningful approach to preventing degenerate corruption processes, supported by the comparison with D3PM-Reinforce and FLDD.
-
VSDD shows improvements in molecular generation and playlist generation while preserving diversity.
Critical weaknesses
-
Only applicable to small vocabulary sizes, with a discrete-time Markov chain (DTMC) setting. This is the main concern that seems difficult for the authors to solve. Constructing requires storing a matrix, and further calculating involves matrix multiplications. This is definitely not a scalable choice, which limits its application to domains that require larger vocabulary sizes (i.e., language). Accordingly, the main results only include small-scale experiments. Furthermore, the method is only applicable to the DTMC setting, which is an outdated strategy right now, and extending it to continuous time seems nontrivial.
-
Equations are really hard to follow. This is not about the explanations. The equations themselves are contaminated, making them very confusing for the reader. For example, Equations (5) and (7) omit the exact inputs to the loss ( and are missing). Equation (11) already stops all gradients from flowing through , making the explanations and stop-gradient operations in Equations (7) and (8) redundant.
Questions for authors
-
Could the authors report training runtime, peak GPU memory, and total GPU-hours for VSDD and the learnable-noise baselines? The construction and virtual follower updates may substantially increase computational cost. (W1)
-
Please explain the equations more rigorously. (W2)
-
What is the validation batch in Algorithm 1? Can you clarify whether the validation data used for leader optimization overlaps with the reported evaluation data?
-
Can you clarify the t of in Figure 2? Considering that the is constructed from and , this is not a surprising results...
-
The authors did not mention prior work in the DLM literature related to re-drafting and learning the forward process for masked diffusion (equivalently, the masking order).
Re-drafting:
[1] Wang, G., Schiff, Y., Sahoo, S. and Kuleshov, V., 2026. Remasking discrete diffusion models with inference-time scaling. Advances in Neural Information Processing Systems, 38, pp.147282-147339.
[2] Kim, J., Kim, S., Lee, T., Pan, D.Z., Kim, H., Kakade, S. and Chen, S., 2025. Fine-tuning masked diffusion for provable self-correction. arXiv preprint arXiv:2510.01384.
[3] Schiff, Y., Belhasin, O., Uziel, R., Wang, G., Arriola, M., Turok, G., Zilberstein, R., Elad, M. and Kuleshov, V., 2026. Learn from your mistakes: Self-correcting masked diffusion models. arXiv preprint arXiv:2602.11590.
MDM forward process:
[4] Jing, S., Wu, C., Liu, C., Bahadorinejad, A., Qiao, F., Liu, D. and Geng, T., InfoDLM: an Information-Adaptive Framework for Discrete Diffusion Language Model Pretraining. In Forty-third International Conference on Machine Learning.
[5] Hong, C., Lee, S. and Ye, J.C., 2026. Unifying masked diffusion models with various generation orders and beyond. arXiv preprint arXiv:2602.02112.
6 comments
Notation/Equations. We omitted and from the loss arguments where they were clear from context to keep the notation simple. Both denoising losses depend on the sampled noisy inputs, as already shown in the posterior terms of Equations (5) and (7). We will write these arguments explicitly in the revised manuscript to avoid ambiguity.
Figure 2. We will add the timestep information to the caption (for this figure we used t=25). Regarding the correlation, the transition probabilities are computed from the learned bilinear scores , not directly from cosine similarity. The learned transformation can therefore emphasise or suppress different relationships in the embedding space; a positive correlation with the original cosine similarities is not imposed by our parameterisation. Figure 2 illustrates the structure of the learned process, rather than serving as the main evidence for its effectiveness, and you can think of A as some sort of learned similarity metric in the denoiser's semantic space.
To test this distinction directly, we also ran an additional molecular experiment with , so that the kernel uses cosine similarity without learning the bilinear transformation. Under the same experimental setup, this achieves 73.2% validity and 0.8565 diversity, compared with 84.3% validity and 0.8679 diversity for VSDD. Thus, embedding similarity already provides useful structure, but learning how to use that structure improves both validity and diversity over the cosine-only variant. We will include this ablation alongside the visual analysis.
Related work. We thank the reviewer for pointing us to these additional papers. We are happy to expand our related-work section and explain how these approaches relate to ours. We will include this discussion in the revised manuscript.
Thank you to the authors for the detailed rebuttal, the newly proposed factorized parameterization, and the additional experiments. While the low-rank factorization and the ablation are helpful steps in the right direction, several critical issues remain unresolved:
- Missing diversity metrics in the new 32k-vocabulary TinyStories experiment
In the new TinyStories experiment, you report only generative GPT-2 perplexity (63.68 for VSDD vs. 79.41 for MDLM and 90.27 for D3PM-Uniform) without reporting the corresponding Distinct-1 and Distinct-2 diversity metrics (unlike Table 2a for V=2048). In discrete diffusion, a collapsed transition kernel that repeatedly generates high-frequency tokens artificially lowers generative perplexity by sacrificing output diversity. Reporting generative perplexity alone without diversity metrics is insufficient to verify generation quality on the 32k-vocabulary setting.
- Incomplete computational overhead and peak GPU memory comparisons
Uniform/transition-matrix formulations (such as D3PM-Uniform / UDLM) typically require roughly 2x the peak GPU memory of MDLM, which directly restricts the maximum feasible batch size on limited hardware. In your response, you report 50 GB / 80 GB peak memory only for VSDD on a 50K vocabulary, without providing the corresponding peak GPU memory for the baselines (MDLM, D3PM-Uniform, D3PM-Reinforce, and FLDD) under the exact same batch size and vocabulary configuration. Please report a complete side-by-side comparison of peak GPU memory and training runtime across VSDD and all baselines.
- Unupdated manuscript and missing explanation regarding redundant stop-gradient (sg) operations
Note that current manuscript has not yet been updated with the promised revisions, and my second point in W2 / Q2 was not addressed in the rebuttal (equation 11). Please clarify this in the text and incorporate all promised revisions into the updated PDF.
Overall, I still think the paper needs further strengthening to meet the bar for acceptance, so I am maintaining my current rating. I thank the authors for their extensive rebuttal and hope these suggestions help improve the work for a future submission.
We would like to thank the reviewer for their feedback and the suggestions. Below we address the questions raised by the reviewer.
Scalability. We appreciate the reviewer raising this question. To address the reviewer’s scalability concern, we introduced a factorised parameterisation of Mt (see Equation (3)):
Mt=UCtW,
where U∈RV×r, Ct∈Rr×r, and W∈Rr×V, with r<<V. All three matrices are nonnegative and row-stochastic, so their product remains a valid corruption kernel. To keep the transition process semantics-aware, we compute
U=softmax(EθA),W=softmax(BEθ⊤),
where Eθ contains the denoiser’s embeddings. Importantly, this also addresses the reviewer’s concern about cumulative matrix multiplication. Sharing U,W across time steps gives an exact representation Qt=αtI+UHtW, where Ht∈Rr×r is updated recursively using small matrices. Consequently, neither the individual transitions nor their cumulative products require storing V×V matrices. Their storage scales as O(Vr+Tr2).
Using this factorisation, we carried out an additional TinyStories experiment with a 32k vocabulary, instead of the original 2,048 tokens. VSDD achieves generated-text perplexity 63.68, compared with 79.41 for MDLM and 90.27 for D3PM-Uniform which corresponds to reductions of approximately 19.8% and 29.5% respectively. Thus, the additional experiment directly addresses the vocabulary limitation and strengthens our text-generation results. This factorised version generalises our original approach, so that when U=W=I, r=V and Ct=Mt,ϕ we recover our original kernel in Equation (4) exactly. We will include the factorised parameterisation and the new results in the paper.
Discrete vs. Continuous time. We do not share the belief that discrete-time diffusion is “outdated” and would appreciate clarification of the specific limitation that the reviewer is referring to. The vocabulary-scaling bottleneck comes from how transitions are represented, not from whether time is discrete or continuous. Moving to continuous time does not automatically resolve this: a dense V×V rate matrix would still require quadratic storage. Our factorisation approach solves this problem directly by reducing storage for both single-step and cumulative transitions . The additional 32k-vocabulary experiment demonstrates that this can be achieved within the discrete-time formulation, without requiring a continuous-time extension.
Training overhead. For the profiled configuration with (K=4) probes and (N=100) follower steps, we measured the following training times at identical batch sizes:
The approximately 14% overhead breaks down into around 7% from learned transition-matrix computations and 6% from leader updates.
For this configuration, each leader update uses (K+2) additional denoiser forward–backward passes: (K) for the probe gradients and two for validation-gradient computations. Amortised over (N) follower steps, this gives approximately ((K+2)/N=6%) additional denoiser computation, consistent with the measured leader-update overhead. We will extend the computational comparison with peak GPU memory, total GPU-hours, and measurements for D3PM-Reinforce and FLDD.
When training a language model with 50K vocab size, the peak memory usage on a single A100 GPU was 50Gb out of 80Gb.
Validation data. The validation minibatch in Algorithm 1 is used to evaluate candidate corruptions through the virtual follower updates and their rewards. In the playlist experiments, these playlists are strictly separate from those used to report NDCG@3 and repair rates. The evaluation playlists are not used for either follower training or leader optimisation. Our use of “validation” for both sets made this distinction unclear; we will explicitly describe their separate roles in the text.