acceptodds
Under review as a conference paper at ICLR 2027

Non-Markovian Uniform-State Diffusion Language Models

Abstract

Uniform discrete diffusion language models (UDLMs) are typically built on a fixed Markov coupling between noise levels, which jointly determines their reverse dynamics, training objective, sampling behavior, and likelihood evaluation. We introduce NoMUD, a family of non-Markovian UDLMs that makes this coupling explicitly designable through two degrees of freedom while preserving the same marginals. Despite inducing different transition dynamics, all feasible couplings share the same optimal predictor, allowing the coupling to be adapted during inference. We exploit this freedom in three settings: optimizing the coupling tightens the negative evidence lower bound for likelihood evaluation; confidence-aware coupling enables adaptive retention, correction, and exploration during sampling; and a learnable coupling improves the flexibility of few-step distillation. Together, these results demonstrate that explicitly designing the coupling provides a simple and effective way to enhance the flexibility of UDLMs and unlock their broader potential.

Primary area:
generative models
Keywords:
Diffusion language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UcSwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a non-markovian diffusion process (nomud), which operates via two parameters, lambda and sigma. These two parameters control the coupling in the noise levels, as described in Eq 5. By changing the values of lambda and sigma, the authors are able to get a variety of qualitatively different behaviors from the sampler, such as being an 'absorbing' transition.

Critical strengths

The design space of lambda/sigma variables is fairly interesting and seems to have a fairly major impact on the empirical behavior of the models.

The experiments seem reasonable, with the design of the experiments matching those of prior works. That said, diffusion PPL evaluations always seem a bit tricky.

The writing is pretty clear, and the authors present their idea in fairly intuitive ways.

Critical weaknesses

Diffusion PPL evaluations have always seemed a bit tricky to me, as they involve comparisons of two lower bounds, and the evaluations can measure either the quality of the model or the tightness of the bound. In some cases, we can ignore this by saying that the bounds are the same for two models, and we expect the gap to be similar, but I'm not sure that's the case here, given that we have 2 extra parameters that we are (implicitly) optimizing over as hyperparameters.

Additionally, I think the results in Table 2 look good, but I also think that the avg win gap is maybe being driven by PTB, which has a strong 12-point gap for NoMUD. I think given that NoMUD has this additional hyperparameter and design space, we'd want NoMUD to win by more?

Finally, it seems a bit strange to me that NoMUD Rand is the one that does significantly better than all others. Part of the appeal of the method to me was that you could come up with more clever couplings, but if we're just uniformly picking within the feasible range, then it doesn't really seem that we take advantage of that. We could appeal to pure performance arguments, but I think given my second point above, it's not clear if this is a win on pure performance alone.

Questions for authors

My comments above are also questions - perhaps I have missed something in the comments above, and would appreciate clarifications from the authors.

4 comments

Thank you for the valuable feedback, here's our response to the concerned problems.


Q1 & Q2. The likelihood evaluation protocol and the effectiveness of our method.

A1 & A2 The NELBO of a discrete diffusion model depends on both the predictor and the variational bound. For a given coupling, the local objective averaged over XtX_t decomposes as follows (Appendix B.2.2 in paper).

=I(X_0;X_s\mid X_t) +\mathbb E_{X_t}D_{\mathrm{KL}}\left(q(X_s\mid X_t) \mid p_\theta(X_s\mid X_t)\right).$$ The first term is the conditional mutual information determined by the reference coupling. The second measures the discrepancy between the true and model reverse transitions. In general, changing the coupling can affect both the variational decomposition and the fit of the reverse transition. In a Markov UDLM, a fixed coupling associates the predictor with a particular generative process. NoMUD uses randomized-coupling training to obtain a shared predictor that supports a family of processes. We define oracle coupling-optimized PPL as the PPL obtained when the model is evaluated under its optimal non-Markovian process. It measures the variational performance of the shared predictor under its optimized non-Markovian coupling. Section 3.3 gives the definition, and Section 4.1 reports the evaluation. To isolate prediction quality, we also evaluate the predictor as a clean-token classifier. NoMUD-Rand outperforms DUO at most evaluated timesteps (Figure 3 in paper). PTB accounts for most of the improvement (Table 2 in paper). Its financial-news text differs from the OpenWebText corpus used in training. The larger gain on PTB shows that our coupling optimization has a greater effect under distribution mismatch. Beyond PTB, NoMUD (Rand, Opt) improves on DUO on four of the remaining six datasets. To assess the practical value of the score beyond the absolute NELBO, we added the following two experiments to Section 4.1. **(a) Downstream multiple-choice tasks.** In these tasks, likelihood scores compare the relative probabilities of candidate answers, so their relative ordering matters. We applied training-free oracle coupling optimization to pretrained 1.7B DUO and scored each candidate with the model weights fixed. As shown in Table 1, **accuracy improves on 4/5 tasks, with an average absolute gain of 1.07%.** The optimized scores better distinguish the candidate answers on these tasks. **Table 1. Multiple-choice accuracy with pretrained 1.7B DUO.** |Task|DUO (Markov)|DUO (Opt)|$\Delta$ Acc (%)| |-|-|-|-| |ARC-Easy|54.92%|**55.18%**|+0.25| |OpenBookQA|19.40%|**22.00%**|+2.60| |PIQA|63.17%|**63.55%**|+0.38| |RACE|34.93%|**37.32%**|+2.39| |SocialIQA|39.15%|38.89%|−0.26| |**Avg**|**42.31%**|**43.39%**|**+1.07**| **(b) Monte Carlo stability.** Likelihood ratios are also used to estimate importance weights in reinforcement learning. When the compute budget limits the number of MC draws, estimation variance affects the stability of those weights. We used the 1,024-draw estimate as the reference mean and filtered extreme values using a 20-nat threshold. Table 2 compares the log-score variance with 1, 2, 4, 8, and 16 draws. **Variance falls at every budget, with a mean reduction of 14.73%.** Together, the experiments show the benefits of oracle scoring for candidate ranking and small-budget estimation. **Table 2. Variance of the Monte Carlo log score (nats²).** |MC draws|Opt variance|Markov variance|Reduction| |-|-|-|-| |1|1.995|2.351|**15.14%**| |2|1.002|1.178|**14.98%**| |4|0.498|0.585|**14.93%**| |8|0.251|0.294|**14.63%**| |16|0.126|0.147|**13.99%**|

Q3. Randomized couplings in training

A3. We train the predictor across couplings, then choose the coupling for the inference task (train for all couplings, inference at adaptive couplings). Feasible couplings share an optimal clean predictor but induce different NELBOs (Proposition 2 and Eqs. 11–12 in paper). Randomized-coupling training exposes the predictor to the diverse reverse transitions in the feasible family and reduces its dependence on any single coupling-specific objective. This training strategy is one of our contributions.

We use confidence-aware coupling for sampling, oracle coupling optimization for likelihood evaluation, and a learnable coupling head for few-step distillation. Randomized training supplies the shared predictor for these designs.

As Table 3 shows, both randomized-coupling training and confidence-aware sampling improve generation. We added this ablation to Appendix C.6.1.

Table 3. Controlled training and sampling ablation. Each entry gives GenPPL.

Training / sampling81632641282565121024
Markov / Markov149.3394.8375.7374.6668.1268.8069.4669.89
Markov / Conf139.0988.1973.4665.9563.6867.5961.9463.47
Rand / Markov139.2581.7467.2161.6158.9159.2659.2459.49
Rand / Conf134.2377.3560.5657.0757.4050.9654.9053.25

Learning the training coupling adds complexity but gives little improvement. We also tested learning the coupling during training. Inspired by learning-order ARMs [1], we learn the coupling parameters that minimize the NELBO. We use a clean head conditioned on x0x_0 to choose the token-wise variational coupling (λt,σt)(\lambda_t,\sigma_t) and a reverse head conditioned on the hidden representation of xtx_t to predict the generative coupling. We minimize the NELBO under the same training budget. Tables 4 and 5 give the results.

Table 4. GPT-2 Large GenPPL with confidence-aware sampling.

Training / steps81632641282565121024
Randomized coupling134.2377.3560.5657.0757.4050.9654.9053.25
Learned training coupling128.8579.4264.2658.9454.7453.6055.6058.21

Table 5. Oracle coupling-optimized PPL for randomized and learned training couplings.

Eval PPLPTBWikiTextLM1BLAMBADAAGNewsPubMedArXivAvg
Randomized coupling73.5536.3675.3157.6872.1650.9443.3358.48
Learned training coupling74.2036.7775.1760.3773.0349.1641.5158.60
Δ\Delta+0.89%+1.15%−0.18%+4.66%+1.20%−3.50%−4.19%+0.22%

The learned training coupling improves likelihood evaluation on three of the seven datasets, but the arithmetic mean PPL rises slightly from 58.48 to 58.60, and generation quality declines overall. We added the design and both comparisons to Appendix C.6.2.

Reference

[1] Wang et al. Learning-Order Autoregressive Models with Application to Molecular Graph Generation. ICML, 2025.

I wanted to comment and say that I appreciated the author's rebuttal, which I think generally adds to the understanding of the paper. I don't think this quite moves it up to a 4 for me (I think that'd just be a big jump that's rarely justified in rebuttal) but I continue to be positive about this paper.

Thank you for your response and positive assessment. Your suggestions are highly valuable for improving the experimental presentation of our paper. We will ensure that the corresponding updates are reflected in Section 4.1 and Appendix C.6. Thank you again for the time and care you have put into this discussion.

Reviewer_S3U6OpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes NoMUD, a framework for designing non-Markovian couplings in uniform discrete diffusion language models. It keeps the prescribed noisy marginals fixed while separating clean-token prediction from the transition dynamics used to update the noisy state. The authors instantiate this via randomized-coupling training, confidence-adaptive sampling, likelihood evaluation, and consistency distillation.

Critical strengths
  1. A clear and interesting modeling perspective. Separating denoising predictions from transition dynamics provides a useful way to organize the design space of uniform diffusion models. The coupling formulation connects retention, correction, and exploration within a common framework.

  2. Analytically structured transition design. Rather than introducing unconstrained sampling heuristics, the paper derives feasible couplings and their associated reverse transitions. The shared-optimum result also offers a theoretical motivation for reusing a clean-token predictor across different couplings.

  3. Multiple applications of the framework. The paper explores sampling, training, evaluation, and distillation. The cross-sampler experiments and coupling-head ablation are useful steps toward understanding the contribution of individual components.

Critical weaknesses
  1. Practical scope is problematic. The method is only evaluated on uniform-corruption DLMs, while masked/absorbing DLMs are the more mainstream models.

2. . Likelihood evaluation lacks a clear model identity. Eq. (13) optimizes the coupling separately for each clean example and time step. The paper should state explicitly which single generative distribution is being bounded after this per-example optimization. If the procedure instead measures oracle, example-dependent adaptation of the reverse model, it should be presented as such and not directly compared with fixed-model perplexity.

3. Improvements are not cleanly attributed. NoMUD changes the reverse parameterization, training couplings, confidence-adaptive sampling, and an additional coupling head for distillation. Comparisons against Duo therefore do not isolate how much gain comes from the coupling framework itself. Controlled experiments are needed: same architecture and posterior parameterization, fixed Markov vs. randomized-coupling training, same sampling budget, and confidence-adaptive coupling vs. simpler confidence-based resampling baselines.

  1. Empirical evidence for practical benefits is limited. The experiments use 170M-parameter models on OpenWebText and evaluate generation mainly with GPT-2 Large perplexity and unigram entropy. These metrics give limited evidence about semantic coherence, factuality, instruction following, or useful diversity. The paper would be stronger with a targeted correction benchmark, downstream evaluation, quality-matched wall-clock comparisons, and multiple independent training runs to establish the robustness of smaller gains.
Questions for authors

NAN

8 comments

Thank you for the valuable feedback, here's our response to the concerned problems.


Q1. Why focus on UDLMs rather than MDLMs?

A1.

  1. UDLMs support few-step sampling [1] and iterative self-correction [2], and have shown promise at scale [3]. This makes uniform diffusion a worthwhile direction in which to study how non-Markovian processes can improve training and generation.
  2. Our framework also covers non-Markovian masked diffusion. One degree of freedom remains in the masked case, corresponding to the probability of remasking a decoded token. This recovers the remask-and-resample update in ReMDM [4]. We made this connection explicit in Appendix B.7.1.

Q2. Clarification of the likelihood evaluation protocol.

A2. In the original Markov setting, the fixed coupling associates a predictor with one generative process. In NoMUD, randomized-coupling training allows one predictor to support a family of generative processes. We define oracle coupling-optimized PPL as the PPL obtained when the model is evaluated under its optimal non-Markovian process. The resulting PPL evaluates the shared predictor under oracle adaptation within the non-Markovian family. Section 3.3 gives the definition, and Section 4.1 reports the evaluation. We claim an improvement in the estimator, without claiming a lower evaluation PPL for a fixed generative model. Two experiments demonstrate the practical advantages of our estimator.

(a) Downstream multiple-choice accuracy. One use of likelihood scoring is to distinguish more plausible candidates from less plausible ones. For a question with nn candidate answers xix_i, the model compares estimates of log⁡p(xi)\log p(x_i), so their relative ordering matters. Given our compute budget, we used pretrained 1.7B DUO and applied oracle coupling optimization without additional training. As shown in Table 1, oracle scoring improves accuracy on 4/5 tasks, with an average absolute gain of 1.07%.

Table 1. Multiple-choice accuracy with pretrained 1.7B DUO.

TaskDUO (Markov)DUO (Opt)Δ\Delta Acc(%)
ARC-Easy54.92%55.18%+0.25
OpenBookQA19.40%22.00%+2.60
PIQA63.17%63.55%+0.38
RACE34.93%37.32%+2.39
SocialIQA39.15%38.89%−0.26
Avg42.31%43.39%+1.07

(b) Lower variance with few Monte Carlo draws. Likelihood ratios between models are used in reinforcement learning to estimate importance weights. With only a few Monte Carlo (MC) draws, the stability of the estimate matters. We used the 1,024-draw estimate as the reference mean, evaluated the variance of the log score with 1, 2, 4, 8, and 16 draws, and filtered extreme values using a 20-nat threshold. We optimized the coupling using only the draws available at each budget. As shown in Table 2, oracle scoring reduces variance at every budget, by 14.73% on average.

Table 2. Variance of the Monte Carlo log score (nats²).

MC drawsOpt varianceMarkov varianceReduction
11.9952.35115.14%
21.0021.17814.98%
40.4980.58514.93%
80.2510.29414.63%
160.1260.14713.99%

Q3. Improvements that were not cleanly attributed

A3. Our controlled ablation shows that randomized-coupling training improves on fixed Markov training, and confidence-aware sampling improves on both Markov sampling and top-kk confidence resampling.

To isolate these effects, we used the same clean-posterior parameterization (Eq. 8 in paper), architecture, and sampling budget. We trained the fixed Markov model with λ=1/V\lambda=1/V and σ=1\sigma=1. For each predictor, we compared Markov sampling, confidence-aware sampling, and a top-kk confidence baseline adapted from [5]. At each of KK steps, the baseline resamples the 1.2×L/K1.2\times L/K tokens with the largest confidence gaps (pmax⁡−pmp_{\max}-p_m, the difference between the maximum predicted token probability and the probability of the current token) and retains the others. Table 3 gives the results.

Table 3. Controlled training and sampling ablation. Each entry gives GenPPL.

Training / Sampling81632641282565121024
Markov / Markov149.3394.8375.7374.6668.1268.8069.4669.89
Markov / Conf139.0988.1973.4665.9563.6867.5961.9463.47
Markov / Top-k289.48186.00144.89119.29102.5191.1882.1170.90
Rand / Markov139.2581.7467.2161.6158.9159.2659.2459.49
Rand / Conf134.2377.3560.5657.0757.4050.9654.9053.25
Rand / Top-k283.87177.56125.36100.8789.3576.4362.0859.34

With the parameterization fixed, we attribute the improvement from Markov training and sampling (M&M) to randomized training and confidence-aware sampling (R&C). We average the gains along the two paths, M&M → R&M → R&C and M&M → M&C → R&C. Writing PMM,PRM,PMC,PRCP_{MM},P_{RM},P_{MC},P_{RC} for the corresponding GenPPL values, the training and sampling contributions are

\qquad r(C)=1-r(R).$$ Randomized training accounts for roughly 70% of the gain at 32–256 steps, and about 60% at the other budgets. Confidence-aware sampling accounts for the remainder. Both improve generation quality, and their combination outperforms the heuristic confidence sampler. We added the M&M-to-R&C experiment, the contribution breakdown, and the top-$k$ results to Appendix C.6.1. --- **Q4. Practical benefits.** **A4.** **Downstream evaluation.** As discussed in Q2, the oracle likelihood score improves candidate ranking and gives lower-variance estimates with few MC draws. We added both experiments to Section 4.1. **Quality-matched wall-clock time.** We measured the number of NoMUD steps needed to match DUO and RADD at 64, 256, and 1,024 steps, within 2% in GenPPL and 0.02 nats in unigram entropy. We used a grid search over step counts and retained the temperature settings from our previous generation experiments. As shown in Table 4, **NoMUD takes less time per batch at all six matched operating points.** We added the timing results to Appendix C.2. **Table 4. Wall-clock time at matched generation quality.** |Baseline|Baseline steps|NoMUD steps|Baseline wall-clock time (s/batch)|NoMUD wall-clock time (s/batch)| |-|-|-|-|-| |DUO|64|38|10.379|3.973| |DUO|256|47|46.841|4.939| |DUO|1024|38|190.446|3.973| |RADD|64|24|3.660|2.551| |RADD|256|30|14.917|3.080| |RADD|1024|27|61.761|2.777| **Multiple training seeds.** Each complete training run takes two to three days. Given our compute budget, we evaluated independently seeded randomized-coupling runs after 20,000 training steps. Tables 5 and 6 show that evaluation loss and generation quality are robust across the tested seeds at this checkpoint. We added both results to Appendix C.6.3. **Table 5. Evaluation loss across training seeds at 20,000 steps.** |Dataset|Mean loss|Std| |-|-|-| |PTB|4.962|0.027| |WikiText|4.202|0.025| |LAMBADA|4.399|0.041| |AGNews|5.071|0.023| **Table 6. Generation across five training seeds at 20,000 steps (mean ± Std).** |Steps|GPT-2 Large GenPPL|Entropy (nats)| |-|-|-| |128|98.86 ± 4.01|5.491 ± 0.012| |256|96.69 ± 4.39|5.481 ± 0.012| |512|93.57 ± 4.31|5.463 ± 0.012| |1024|94.40 ± 4.90|5.468 ± 0.015| **References** [1] Sahoo et al. The Diffusion Duality. ICML, 2025. [2] Deschenaux et al. The Diffusion Duality, Chapter II: $\Psi$-Samplers. ICLR, 2026. [3] von Rütte et al. Scaling Behavior of Discrete Diffusion Language Models. ICLR, 2026. [4] Wang et al. Remasking Discrete Diffusion Models with Inference-Time Scaling. NeurIPS, 2025. [5] von Rütte et al. Scaling Behavior of Discrete Diffusion Language Models. ICLR, 2026.

Thank you for the response.

  1. On the first point, uniform-state models can decode in fewer steps, but masked DLMs can also accelerate sampling through remasking and block-based decoding. It is unclear that this advantage is specific to the uniform formulation.

  2. A stronger case would require scaling parameters and data to a usable model. 170M is indeed too small to be useful.

And other main concerns have been solved for me.

Reviewer_5u9WOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes NoMUD, a non-Markovian uniform discrete diffusion language model. This change effectively conditions the forward transition model on x0x_0 in addition to the previous state xsx_s via a two-parameter coupling (λ\lambda and σ\sigma). The NELBO, neural parameterizations, and confidence-aware sampling are largely repurposed from existing diffusion language models. Evaluation is done via likelihood and perplexity via GPT-2 tokenization. SEDD and RADD serve as masked-diffusion baselines, USDM, Duo, SDDLM serve as uniform diffusion baselines.

Critical strengths

The proposed approach is sensible and seems to work well empirically, providing a clear improvement in the flexibility and capacity of diffusion language models. It also unifies prior work, recovering the Markov UDLM and absorbing as special cases. The difference from masked diffusion is well-explained.

While I am not fully up-to-date on evaluation of diffusion language models, the experiments seem well-done in depth and breadth.

Critical weaknesses

Relationship to other discrete generative models: Campbell et al. (ICML 2024, "Generative Flows on Discrete State-Spaces") and Gat et al. (NeurIPS 2024, "Discrete Flow Matching") are not cited. I believe they also show marginal-preserving couplings, a shared optimal predictor, and couplings chosen at inference.

Questions for authors
  1. How does NoMUD's two-parameter family relate to the marginal-preserving rate-matrix family in Campbell et al. (2024) for uniform noise?

  2. Which prior method corresponds to the "one degree of freedom" special case mentioned in the introduction?

3 comments

Thank you for the valuable feedback, here's our response to the concerned problems.


Q1 and W1. Connections between NoMUD and the work of Campbell et al. and Gat et al.

A1. Thank you for your thoughtful review and for pointing out the connection to discrete flow matching. The following are the similarities and differences between NoMUD and related works.

Similarity. All three works study how to design couplings with two degrees of freedom while preserving prescribed marginal distributions (the probability path). For uniform noise, Campbell et al. [1] construct a family of marginal-preserving rate matrices. Under token symmetry, two degrees of freedom remain. Let τ=αt\tau=\alpha_t, β=1−τ\beta=1-\tau, and r=Vτ+βr=V\tau+\beta. In Campbell et al.’s notation, the detailed-balance part of NoMUD’s infinitesimal rates corresponds to

b_\tau=\frac{\lambda_t}{\tau},\qquad c_\tau=\frac{\sigma_t}{V\tau}.$$ For clean-posterior prediction, all three methods have the true clean posterior as their optimal target. **Difference 1. Finite-time transitions.** Campbell et al. start from an infinitesimal rate matrix and construct marginal-preserving generative processes by evolving a continuous-time Markov chain (CTMC). Their rates specify the transition from $t$ to $t+dt$. We directly give the conditional transition kernel and its feasible region for a finite interval $s<t$. Even after integration, the resulting token-symmetric CTMC kernels remain constrained by the rate-matrix evolution and form a subset of our feasible finite-time kernels. Direct finite-time parameterization gives us a larger design space for long sampling steps. **Difference 2. Training objective and analysis.** Campbell et al. and Gat et al. use clean-posterior cross-entropy to learn the predictor. NoMUD starts from the exact reverse transition, constructs the model reverse transition, and derives a coupling-dependent NELBO from the KL divergence between them. In short, DFM directly matches the clean posterior, while we match the reverse transition. With our training loss, NoMUD achieves lower evaluation and generation PPL than the CE-trained SDDLM in our comparisons (Tables 2 and 3 in paper). **Difference 3. Coupling design for different tasks.** Campbell et al. vary the inference-time rates to control CTMC stochasticity. Gat et al. study probability schedulers and corrector sampling. In NoMUD, we systematically study coupling design. We (a) randomize the coupling during training to expose a shared predictor to different transitions, (b) optimize the coupling for likelihood evaluation, and (c) combine Absorb, Flip, and Explore according to confidence during sampling. To further evaluate the effectiveness of our method, we tested confidence-aware sampling on the pretrained 1.3B uniform-source DFM checkpoint released with FS-DFM [4]. We use the same clean-posterior parameterization and compare both samplers at matched entropy. As shown in Table 1, **confidence-aware sampling improves GenPPL at all eight budgets, with a mean reduction of 13.31%.** We added the connection to DFM to Section 5 and the detailed comparison and transfer experiment to Appendix B.7.4. **Table 1. GPT-2 Large GenPPL on the frozen 1.3B DFM checkpoint at matched entropy.** |Sampler / steps|8|16|32|64|128|256|512|1024| |-|-|-|-|-|-|-|-|-| |DFM baseline|217.11|95.44|64.97|54.24|52.85|48.94|48.01|47.89| |Confidence-aware|**186.81**|**82.11**|**57.52**|**48.05**|**42.73**|**43.16**|**43.11**|**40.94**| --- **Q2. One-degree-of-freedom special case** **A2. The special case is the categorical construction in Appendix A of DDIM [5].** Let $\eta$ be the weight that DDIM assigns to the current token. Its reverse kernel has probabilities $\eta x_t+(\alpha_s-\eta\alpha_t)x_0+(\beta_s-\eta\beta_t)\mathbf u$. Moving from the clean token to a given wrong token and moving between two distinct wrong tokens both receive mass $(\beta_s-\eta\beta_t)/V$. Equating these entries in the NoMUD reverse kernel gives $$V\beta_t\lambda_{t,s}=r_t\sigma_{t,s}.$$ Thus DDIM varies one parameter along a line segment in NoMUD’s two-parameter family. We added the explicit correspondence to Appendix B.7.3. **References** [1] Campbell et al. Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design. ICML, 2024. [2] Gat et al. Discrete Flow Matching. NeurIPS, 2024. [3] Zhu et al. Simple Denoising Diffusion Language Models. 2025. [4] Karimi Monsefi et al. FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models. ICLR, 2026. [5] Song, Meng, and Ermon. Denoising Diffusion Implicit Models. ICLR, 2021.

Thanks for the detailed reply. Assuming the discussion of Campbell et al. and Gat et al. is added to the camera-ready, I support acceptance.

Thank you for revisiting our response and for supporting acceptance. Your suggestion to discuss Campbell et al. and Gat et al. helped us place NoMUD more accurately within the existing literature and clarify its contributions. We have added this discussion to Section 5 and Appendix B.7.4 of the revised manuscript. We appreciate the time and care you have devoted to reviewing our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.