acceptodds
Withdrawn from ICLR 2027

Mixing Times of Glauber Dynamics on Masked Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Masked language models (MLMs) define local conditional distributions over tokens but do not, in general, correspond to any consistent joint distribution over sequences. This raises a fundamental question: what global distributional behavior is induced when such conditionals are used iteratively for generation? We address this question by modeling iterative masked-token resampling as a Glauber dynamics Markov chain on the discrete space of token sequences. We first show that MLM conditionals are intrinsically incompatible: we introduce a rectangle test that certifies this incompatibility and empirically verify its prevalence across modern MLMs. We then provide a theoretical analysis of the induced Markov chain. Under bounded cross-token influence, we establish a high-temperature contraction result implying mixing time where is the sequence length. In contrast, we prove that under a uniform local margin condition, the chain exhibits metastability, with exponentially slow escape from semantic basins at low temperatures. Empirically, we demonstrate a phase transition in mixing behavior as a function of temperature and sequence length, consistent with the theoretical predictions. We further characterize the induced stationary behavior through semantic trajectories, identifying persistent structures such as long-lived traps and recurrent semantic basins, with political content serving as a measurable case study.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Glauber Dynamics, Markov Chains, Masked Language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LrqNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies BERT models using Glauber dynamics, which re-sample one token at a time, to explore what distribution would be sampled by these dynamics over long time horizons. A number of results are derived from this dynamical systems perspective. First, it is shown the transitions are not consistent, so that a stationary distribution is not well defined. Second, results on mixing times are derived, showing that slow and fast mixing regimes exist. Finally, an empirical study was done on the meta-stability of the dynamics, showing that the Glauber dynamics slowly drift through different meta-stable basins reflecting different topics.

Critical strengths

I really like the dynamical systems perspective, and found it a fresh way to study transformer models. Analyzing generative dynamics based on some meta-stability property seems like a cool way to probe the dynamics that goes beyond just looking at the transitions. Theoretical analysis of mixing times is also interesting and novel, as other text generative models don't have a way to naturally consider this.

Critical weaknesses

The sampling results aren't well-motivated because the models studied (BERT) are not used for sampling. And the models used for sampling (e.g. masked diffusion models) were not considered, even though I feel they could be (see below).

Because the motivation for studying these dynamics with BERT is not very tight, the results also felt disconnected and therefore had no joint impact. Each individual result was too under-developed to have a strong payoff. I'll discuss some thoughts about the individual results and then speculate on some ways to strengthen the paper.

Order

The rectangle test / part about the order was not very compelling to me because there have been a rather large number of papers that have explored this idea about inconsistency based on order dependence. While they were focused more on the incompatibility of marginals under different orderings for masked diffusion models, I didn't really see a strong reason you couldn't / shouldn't compare with these. Your case is a special case of MDM where you are choosing order of last token conditioned on others. I was thinking of a recent paper, "DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking", but there are many older papers that discuss that too, e.g. I found "Inconsistencies in MLM" arxiv: 2301.00068. Also: Takateru Yamakoshi, Thomas L. Griffiths, and Robert D. Hawkins. Probing BERT’s priors with serial reproduction chains. In Findings of the Association for Computational Linguistics: ACL 2022, 2022.

There was a comment like "stationary distribution is emergent" - this didn't sound like the right way to express this.

Mixing times

My understanding of this result is that you give an upper bound on mixing time (it would be interesting to have a sense of how tight this is - if n log n is generally required it's a quite important reason this approach should not be useful), and then the example of an empirical situation with the "!" process where you get certifiably slow mixing. I guess I didn't understand the "!" process very well, it seems kind of artificial so I didn't get a sense of whether we should expect slow mixing issues generally or only in special circumstances.

Minor comments: 363 had an unresolved ref.

Meta-stability

The meta-stability / topic result with politics seemed the weakest - both in its execution but also with writing and presentation. One issue is that this became kind of a computational social science direction ("sociotechnical" as written in the paper) - which doesn't really match the rest of the paper. And again, it's hard to see the point of this sociotechnical analysis for a sampling method that nobody uses and the paper is not even advocating as a constructive sampling approach.

In this section Table 3 repeats columns, and it was very confusing to understand what you were trying to show here. "Everything drifts towards politics?" I don't think the results obviously support it, and there's not enough development of the experiment for me to trust the way you measure this.

Minor comments: 462 "5. predicts."

Constructive feedback

I don't know what the remedy is, but I can imagine a few possible paths for how this paper could work, through strengthening the motivation, results or ideally both together.

(a) Rather than glossing over the fact that you are studying a kind of sampling no one uses (sampling via Glauber dynamics on BERT models), maybe it would be better to lean into it. To me, what was fascinating about this paper is the idea that BERT dynamics can be used to sample, in principle, via Glauber dynamics. That means that masked diffusion and autoregressive are not the only two possibilities for text sampling. While it seems from your refs that a few others have noticed this, your analysis can maybe provide some insight into why this doesn't work, and evaluate the promise for whether it ever could. (The n log n result, if it were a lower bound, seems to establish a strong reason that it won't be better than AR which takes only n steps to sample.) So the story would be more like - we have a completely different paradigm for sampling with transformers via Glauber / editing dynamics, could it ever be a dark horse that becomes competitive? Then structure and refine the results to answer this question: mixing time is too slow, no well-defined stationary distribution, leading to biased meta-stable states. And you could speculate (even an experiment here?) about if / how a model could be improved to effectively sample with Glauber. (And instead of barely mentioning GGM paper, you could strongly frame Glauber as a serious Gen AI suggestion and directly critique by using your methods to show why this approach is not promising. )

(b) Another angle that would be interesting to engage with is the recent rise of editing-based masked diffusion models. Llada 2.1 (I think) has an editing model. To me, it is a no-brainer to adapt it to your framework and study if it is qualitatively the same or different to BERT models. I really don't see why any MDM model could not be considered (though the editing models are a more interesting fit for your methodology). There's also been some interesting differences between MDM models trained from scratch and MDM models adapted from AR (like DREAM). Again, I feel your proposed methods could give a unique insight.

(c) If the idea is to only study BERT, and not engage with any recent generative model research, it may be possible but then I think you would need to show that some of these results have bearing on real-world use cases for BERT. You say: "we investigate global semantic behavior through topic-level metastability and transition structure in the induced Markov chain" - but it's not clear how global semantic behavior in practical use cases is connected to your results.

(d) A wildly different direction could build on the mixing time result to ask fundamental questions about when efficient sampling could be done using Glauber dynamics, with some inspiration from https://www.pnas.org/doi/10.1073/pnas.2311810121.

Questions for authors

While I generally feel that the changes in motivation and results needed are too significant to change in the rebuttal, I like the idea for the paper, remain open-minded, will engage with the rebuttals and would change my score with compelling enough arguments.

Reviewer_aN31OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a combination of Glauber dynamics and Masked Language Models (MLM), where randomly selected tokens are repeatedly masked and resampled, and study it as a long run MCMC dynamics. The authors asses the trained MLM models for compatibility of its conditional distributions to construct a valid MCMC and come up with rectangle test for that purpose. Then authors provide theoretical analysis of the MCMC for both rapid and slow mixing.

Empirically, authors show how MLM Glauber dynamics mixing depends on a temperature and sequence length. Additionally, they explore the semantics of trajectories and identify the persistent structures such as long living traps and "semantic basins".

Critical strengths
  • The study of MLM induced conditional distributions as a kernel for MCMC is quite interesting and could lead to practical algorithms and insightful analysis.
Critical weaknesses
  • The idea of using the MLM as the kernel in long-run MCMC dynamics is not novel. The paper [1] introduces similar MCMC kernels based on MLM conditional distributions for the long-run dynamics, while [2] also highlights the incompatibility problem and designs a test for it.
  • The manuscript presentation needs polishing, for example:
    • Moreover, Section 5 is written in a technical manner and doesn't provide enough motivation and context for non-MCMC readers
    • Proposition 4 and Theorem 3 lack proofs. They may be trivial, but they still must be either written, referenced, or the names of the results should be changed to corollary or something else.
    • Smaller details:
      • Symbol missing at the start of line 210
      • Broken reference at line 363
      • (τ1)(\tau 1) seems as misprint in line 361
      • ". predicts. " line 462
  • Most of the theoretical results from Section 5 are either trivial or already known in very similar form, which undermines the theoretical contribution of the paper.
  • Given the fact that, from the methodological point of view, the paper is not novel [1, 2], the mixing study seems like the main contribution. But the practical motivation for such analysis is unclear. Why do we need to study these dynamics? While some of the current language diffusion models do utilize similar generation patterns [3], it is not clear how such an analysis helps in practice.
  • The baseline models used for dynamics analysis are outdated. The recently developed family of masked diffusion language [3, 4] models seems more fitting for such an analysis.

References: [1] Yamakoshi, Griffiths & Hawkins, “Probing BERT’s Priors with Serial Reproduction Chains” (Findings of ACL 2022).

[2] Goyal, Dyer & Berg-Kirkpatrick, “Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis–Hastings” (ICLR 2022).

[3] Wang et al., “Remasking Discrete Diffusion Models with Inference-Time Scaling” (NeurIPS 2025).

[4] Sahoo et al., “Simple and Effective Masked Diffusion Language Models” (NeurIPS 2024).

Questions for authors

I would kindly ask authors to clarify the novelty w.r.t. [1] and [2]

Reviewer_6WNcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies iterative single-token resampling in masked language models through Glauber dynamics. It introduces a rectangle test for incompatibility between local conditional distributions and evaluates it across several pretrained MLMs. It then derives sufficient conditions for rapid mixing under bounded cross-token influence and slow mixing under a uniform local margin assumption. Experiments examine temperature-dependent semantic drift and coupling, persistent trap states, and recurrent topic regions. The paper aims to connect local prediction behavior with the global stochastic dynamics induced by repeated masked-token updates.

Critical strengths
  1. A useful formulation and a concrete diagnostic for conditional incompatibility. The paper clearly distinguishes the existence of a stationary distribution for the resampling chain from the existence of a joint distribution compatible with the model’s conditionals. The rectangle test makes incompatibility directly testable, and the results across five MLMs provide evidence that it occurs across the evaluated architectures and model sizes. This is a useful foundation for studying the behavior of iterative masked-token generation.

  2. An informative connection between local predictions and persistent generation dynamics. The influence and margin conditions provide interpretable mechanisms for forgetting initialization and remaining trapped. The long trajectories and catalogue of persistent states complement single-mask analyses and expose behavior that would be difficult to characterize through static predictions alone. These observations make the proposed dynamical perspective worth investigating further.

Critical weaknesses
  1. The claimed empirical certificate of slow mixing is not established. The authors observe positive inward drift at 200 sampled boundary configurations for (N=100), and conclude that escape times are exponential in (N), thereby certifying slow mixing. These observations do not establish a uniform drift condition. Moreover, even positive drift at every configuration on this single boundary level would not, by itself, imply exponentially long escape times. Such a conclusion requires an explicit theorem with appropriate control over the relevant region and specified initial states. Connecting escape times to global mixing also requires an additional argument, such as a bottleneck with controlled stationary mass. Theorem 8 does not supply these missing steps because §5.4 explicitly invokes a different drift argument. This gap directly affects one of the paper’s advertised contributions. The authors should supply the argument and verify its assumptions, or narrow this result to empirical evidence of local stability.

  2. The experiments do not adequately substantiate the claimed mixing-time scaling. An embedding-distance hitting time measures departure from the initial meaning, rather than convergence to stationarity. Likewise, coupling times from sampled MS MARCO initializations characterize those starts and the chosen coupling; they do not establish worst-case total-variation mixing, and failure to couple within a finite budget is not itself a slow-mixing lower bound. Consequently, the (n\log n) fit and temperature heatmap support a qualitative crossover but do not validate the theoretical regimes. Appendix A.1 does not close this gap: its influence estimates are lower bounds on the worst-case quantity, so an estimate below one cannot certify contraction, while the reported slow-mixing lower bounds are numerically uninformative. These limitations do not invalidate the conditional theorems, but leave their applicability to the observed MLM dynamics insufficiently established. Validation in a tractable setting with a computable stationary distribution would help clarify what the diagnostics actually measure.

Questions for authors
  1. Clarification of the slow-mixing certificate: What precise drift theorem justifies the exponential escape-time conclusion in §5.4? Please specify the initial states, the region over which its assumptions must hold, their dependence on (N), and the argument connecting escape times to mixing times. How are these conditions verified by the reported experiment?

  2. Clarification of empirical validation: What argument connects the embedding hitting times and sampled-initialization coupling results to the mixing-time quantity studied in the theorems? If these measurements are intended only as exploratory diagnostics, which claims about mixing-time scaling and phase transitions would the authors retain?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.