- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors introduce a concept of activity, which is basically the expected mismatch cost for a coupling distribution on (X,Y). Minimal activity for the cost 1[X\neqY] gives total variation. The concept is thus tied to maximal coupling theory and optimal transport. The authors suggest that as we sample tokens in discrete diffusion, some sampled tokens change in-between sampling iterations and they propose to couple the samples between consecutive time-steps via maximal coupling to reduce the number of these changes. The authors have empirical result that indeed show that the number of changes is smaller compared to the other sampling schemas. However, it is unclear if the proposed strategy leads to any practical benefit.
Critical strengths
Overall paper is well-written and easy to follow. As someone who is familiar with maximal coupling theory, I found the idea of coupling consecutive steps of a sampler, to be very interesting.
I think overall methodology is clear and makes sense.
The empirical results clearly suggest that the number of token changes significantly decreases when using this method.
Critical weaknesses
I have a few questions / concerns about the paper.
First, I am not sure it is appropriate to attribute the proposed coupling in Section 3.3 to be "Minimal-Activity Finite-time Coupling", because it suggests that this coupling is a new contribution of the paper. The technique the authors use is a very standard technique for constructing maximal coupling, see for example [1]. Therefore, I think it is a bit misleading and the authors should clearly cite the works on maximal coupling and say that the technique is a direct application of [1]. I think the real contribution of the paper is the observation that you can apply this technique to consecutive steps of a sampler.
Second, while the technique reduces number of token changes, it does not seem to lead to practical improvements compared to baselines. How can authors explain this? It basically means that reducing activity does not seem to affect quality of sampling. Then, the question is, why should we do this procedure and why should we care about it? While it is simple, it requires doing additional work and computation and therefore increases the computational cost (For example, you need to compute total variation and you need to resample from the adjusted distribution). Is there a regime where this method is particularly well suited?
nit: For readability, in result tables, can you highlight best metrics in bold and second best with underscore? It'll be much easier to read.
At the moment, I'm inclined to reject, but I am open for discussion.
[1] Jacob, P. Lectures on Couplings and Monte Carlo. https://sites.google.com/site/ pierrejacob/cmclectures, 2021.
Questions for authors
See above.
7 comments
W1: Attribution and the contribution beyond classical maximal coupling
We agree that the common-mass/residual construction is classical maximal coupling. MAFTC names the finite-time sampling construction we use between prescribed conditional marginals.
The original manuscript explicitly credits the classical construction: Section 3.1, p. 4, lines 185–192, Eq. (10), introduces the minimum-activity identity with “By the classical maximal-coupling identity,” citing Griffeath (1975) and Thorisson (2000). Appendix A.1, p. 14, titled “Classical maximal coupling and residual pairing,” describes the family of minimizing couplings. Related Work, p. 9, lines 453–461, also acknowledges earlier applications in discrete generative models and explains that MAFTC applies this principle to prescribed conditional marginals.
Thank you also for recommending Jacob's lecture notes, a useful additional reference on coupling methods. Building on the classical result, the original manuscript makes three contributions:
- Induced versus prescribed transport. Section 3.2, Eqs. (11)–(13), decomposes activity relative to the induced marginal and separately measures its discrepancy from the prescribed marginal . The bound shows why low activity can reflect insufficient marginal transport. Evaluating activity alongside marginal discrepancy distinguishes insufficient transport from reduced coupling excess.
- One-step versus trajectory activity. Section 3.5, p. 6, lines 288–302, distinguishes coupling excess from excess Hamming path length. A trajectory can contain repeated changes even when each constituent coupling is optimal.
- Excess activity in kinetic-optimal dynamics. Theorem 4.2, p. 7, Eqs. (26)–(27), identifies exactly which kinetic-optimal flux contributes activity above the minimum on Gibbs paths. Section 5.1 and Appendix E.3 apply this result to FUDOKI: the endpoint-averaged integrated activities are for the kinetic-optimal dynamics and for the minimum, although every nonzero kinetic-optimal jump decreases distance to the endpoint.
Together, these results explain how the coupling choice affects activity, why low activity alone can be misleading, and where excess arises in existing dynamics. The revised manuscript brings these uses into clearer focus in contribution (ii), the transition from Section 3 to Section 4, and the organization of Section 5.
W2 (1/2): Practical benefits and computational overhead
We understand the concern as whether changing the coupling provides a useful benefit once generation quality and implementation overhead are considered. Even kernels realizing the same prescribed marginals can produce different numbers of changes. Table 6 in the original manuscript shows that MAFTC's trajectory excess decreases as NFE increases, while the baselines' excess increases. Lower Gen-PPL can come at the cost of reduced diversity, so we compare diversity at similar Gen-PPL. Appendix Figure 3 compares Gen-PPL with corpus coverage and within-sequence repetition on the same 2,048 generated sequences per configuration. In the revised manuscript, the trajectory results appear in Figure 1(d) and Table 7, and the quality–diversity curves appear in Figure 2. Section 5.2 explains these complementary aspects of diversity at similar Gen-PPL, and Appendix E.6 gives the corresponding sequence and pooled entropy analysis.
Response to Reviewer s7FA (No. 43815: The Activity Geometry of Discrete Generative Sampling)
Thank you for your constructive comments and for recognizing the clarity of the methodology and the reduction in token changes. We also appreciate your interest in coupling consecutive sampling steps. Your questions about the relationship to classical maximal coupling and the practical benefit after accounting for implementation cost have helped us clarify the contribution and its computational implications. Below, we respond to these questions and describe the supporting evidence and revisions. We use original manuscript for the version reviewed and revised manuscript for the updated version accompanying this response.