- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose a non-markovian diffusion process (nomud), which operates via two parameters, lambda and sigma. These two parameters control the coupling in the noise levels, as described in Eq 5. By changing the values of lambda and sigma, the authors are able to get a variety of qualitatively different behaviors from the sampler, such as being an 'absorbing' transition.
Critical strengths
The design space of lambda/sigma variables is fairly interesting and seems to have a fairly major impact on the empirical behavior of the models.
The experiments seem reasonable, with the design of the experiments matching those of prior works. That said, diffusion PPL evaluations always seem a bit tricky.
The writing is pretty clear, and the authors present their idea in fairly intuitive ways.
Critical weaknesses
Diffusion PPL evaluations have always seemed a bit tricky to me, as they involve comparisons of two lower bounds, and the evaluations can measure either the quality of the model or the tightness of the bound. In some cases, we can ignore this by saying that the bounds are the same for two models, and we expect the gap to be similar, but I'm not sure that's the case here, given that we have 2 extra parameters that we are (implicitly) optimizing over as hyperparameters.
Additionally, I think the results in Table 2 look good, but I also think that the avg win gap is maybe being driven by PTB, which has a strong 12-point gap for NoMUD. I think given that NoMUD has this additional hyperparameter and design space, we'd want NoMUD to win by more?
Finally, it seems a bit strange to me that NoMUD Rand is the one that does significantly better than all others. Part of the appeal of the method to me was that you could come up with more clever couplings, but if we're just uniformly picking within the feasible range, then it doesn't really seem that we take advantage of that. We could appeal to pure performance arguments, but I think given my second point above, it's not clear if this is a win on pure performance alone.
Questions for authors
My comments above are also questions - perhaps I have missed something in the comments above, and would appreciate clarifications from the authors.
4 comments
Q3. Randomized couplings in training
A3. We train the predictor across couplings, then choose the coupling for the inference task (train for all couplings, inference at adaptive couplings). Feasible couplings share an optimal clean predictor but induce different NELBOs (Proposition 2 and Eqs. 11–12 in paper). Randomized-coupling training exposes the predictor to the diverse reverse transitions in the feasible family and reduces its dependence on any single coupling-specific objective. This training strategy is one of our contributions.
We use confidence-aware coupling for sampling, oracle coupling optimization for likelihood evaluation, and a learnable coupling head for few-step distillation. Randomized training supplies the shared predictor for these designs.
As Table 3 shows, both randomized-coupling training and confidence-aware sampling improve generation. We added this ablation to Appendix C.6.1.
Table 3. Controlled training and sampling ablation. Each entry gives GenPPL.
| Training / sampling | 8 | 16 | 32 | 64 | 128 | 256 | 512 | 1024 |
|---|---|---|---|---|---|---|---|---|
| Markov / Markov | 149.33 | 94.83 | 75.73 | 74.66 | 68.12 | 68.80 | 69.46 | 69.89 |
| Markov / Conf | 139.09 | 88.19 | 73.46 | 65.95 | 63.68 | 67.59 | 61.94 | 63.47 |
| Rand / Markov | 139.25 | 81.74 | 67.21 | 61.61 | 58.91 | 59.26 | 59.24 | 59.49 |
| Rand / Conf | 134.23 | 77.35 | 60.56 | 57.07 | 57.40 | 50.96 | 54.90 | 53.25 |
Learning the training coupling adds complexity but gives little improvement. We also tested learning the coupling during training. Inspired by learning-order ARMs [1], we learn the coupling parameters that minimize the NELBO. We use a clean head conditioned on to choose the token-wise variational coupling and a reverse head conditioned on the hidden representation of to predict the generative coupling. We minimize the NELBO under the same training budget. Tables 4 and 5 give the results.
Table 4. GPT-2 Large GenPPL with confidence-aware sampling.
| Training / steps | 8 | 16 | 32 | 64 | 128 | 256 | 512 | 1024 |
|---|---|---|---|---|---|---|---|---|
| Randomized coupling | 134.23 | 77.35 | 60.56 | 57.07 | 57.40 | 50.96 | 54.90 | 53.25 |
| Learned training coupling | 128.85 | 79.42 | 64.26 | 58.94 | 54.74 | 53.60 | 55.60 | 58.21 |
Table 5. Oracle coupling-optimized PPL for randomized and learned training couplings.
| Eval PPL | PTB | WikiText | LM1B | LAMBADA | AGNews | PubMed | ArXiv | Avg |
|---|---|---|---|---|---|---|---|---|
| Randomized coupling | 73.55 | 36.36 | 75.31 | 57.68 | 72.16 | 50.94 | 43.33 | 58.48 |
| Learned training coupling | 74.20 | 36.77 | 75.17 | 60.37 | 73.03 | 49.16 | 41.51 | 58.60 |
| +0.89% | +1.15% | −0.18% | +4.66% | +1.20% | −3.50% | −4.19% | +0.22% |
The learned training coupling improves likelihood evaluation on three of the seven datasets, but the arithmetic mean PPL rises slightly from 58.48 to 58.60, and generation quality declines overall. We added the design and both comparisons to Appendix C.6.2.
Reference
[1] Wang et al. Learning-Order Autoregressive Models with Application to Molecular Graph Generation. ICML, 2025.
I wanted to comment and say that I appreciated the author's rebuttal, which I think generally adds to the understanding of the paper. I don't think this quite moves it up to a 4 for me (I think that'd just be a big jump that's rarely justified in rebuttal) but I continue to be positive about this paper.
Thank you for your response and positive assessment. Your suggestions are highly valuable for improving the experimental presentation of our paper. We will ensure that the corresponding updates are reflected in Section 4.1 and Appendix C.6. Thank you again for the time and care you have put into this discussion.
Thank you for the valuable feedback, here's our response to the concerned problems.
Q1 & Q2. The likelihood evaluation protocol and the effectiveness of our method.
A1 & A2 The NELBO of a discrete diffusion model depends on both the predictor and the variational bound. For a given coupling, the local objective averaged over Xt decomposes as follows (Appendix B.2.2 in paper).
=I(X_0;X_s\mid X_t) +\mathbb E_{X_t}D_{\mathrm{KL}}\left(q(X_s\mid X_t) \mid p_\theta(X_s\mid X_t)\right).$$ The first term is the conditional mutual information determined by the reference coupling. The second measures the discrepancy between the true and model reverse transitions. In general, changing the coupling can affect both the variational decomposition and the fit of the reverse transition. In a Markov UDLM, a fixed coupling associates the predictor with a particular generative process. NoMUD uses randomized-coupling training to obtain a shared predictor that supports a family of processes. We define oracle coupling-optimized PPL as the PPL obtained when the model is evaluated under its optimal non-Markovian process. It measures the variational performance of the shared predictor under its optimized non-Markovian coupling. Section 3.3 gives the definition, and Section 4.1 reports the evaluation. To isolate prediction quality, we also evaluate the predictor as a clean-token classifier. NoMUD-Rand outperforms DUO at most evaluated timesteps (Figure 3 in paper). PTB accounts for most of the improvement (Table 2 in paper). Its financial-news text differs from the OpenWebText corpus used in training. The larger gain on PTB shows that our coupling optimization has a greater effect under distribution mismatch. Beyond PTB, NoMUD (Rand, Opt) improves on DUO on four of the remaining six datasets. To assess the practical value of the score beyond the absolute NELBO, we added the following two experiments to Section 4.1. **(a) Downstream multiple-choice tasks.** In these tasks, likelihood scores compare the relative probabilities of candidate answers, so their relative ordering matters. We applied training-free oracle coupling optimization to pretrained 1.7B DUO and scored each candidate with the model weights fixed. As shown in Table 1, **accuracy improves on 4/5 tasks, with an average absolute gain of 1.07%.** The optimized scores better distinguish the candidate answers on these tasks. **Table 1. Multiple-choice accuracy with pretrained 1.7B DUO.** |Task|DUO (Markov)|DUO (Opt)|$\Delta$ Acc (%)| |-|-|-|-| |ARC-Easy|54.92%|**55.18%**|+0.25| |OpenBookQA|19.40%|**22.00%**|+2.60| |PIQA|63.17%|**63.55%**|+0.38| |RACE|34.93%|**37.32%**|+2.39| |SocialIQA|39.15%|38.89%|−0.26| |**Avg**|**42.31%**|**43.39%**|**+1.07**| **(b) Monte Carlo stability.** Likelihood ratios are also used to estimate importance weights in reinforcement learning. When the compute budget limits the number of MC draws, estimation variance affects the stability of those weights. We used the 1,024-draw estimate as the reference mean and filtered extreme values using a 20-nat threshold. Table 2 compares the log-score variance with 1, 2, 4, 8, and 16 draws. **Variance falls at every budget, with a mean reduction of 14.73%.** Together, the experiments show the benefits of oracle scoring for candidate ranking and small-budget estimation. **Table 2. Variance of the Monte Carlo log score (nats²).** |MC draws|Opt variance|Markov variance|Reduction| |-|-|-|-| |1|1.995|2.351|**15.14%**| |2|1.002|1.178|**14.98%**| |4|0.498|0.585|**14.93%**| |8|0.251|0.294|**14.63%**| |16|0.126|0.147|**13.99%**|