acceptodds
Under review as a conference paper at ICLR 2027

The Activity Geometry of Discrete Generative Sampling

Abstract

Iterative refinement is a key capability of diffusion-based generation and has motivated growing efforts to bring similar behavior to discrete diffusion and flow models. A growing line of work enables such refinement by allowing previously generated tokens to be revised during sampling, thereby shaping the coupling between successive sampling states. Despite its growing role in sampler design, this coupling freedom remains insufficiently characterized. We introduce Activity Geometry to study these finite-step couplings through expected transition cost, which we call activity. We show that activity decomposes into the minimum required by the induced marginal transport and an excess term arising from the coupling. Using this minimum as a reference, we propose Minimal-Activity Finite-Time Coupling (MAFTC) to realize prescribed marginals with minimum activity without retraining or additional model evaluations. In the continuous-time limit, the minimum activity per unit time converges to the minimum expected jump rate charactering the prescribed path. For metric-induced Gibbs paths, we further show that the activity of the kinetic-optimal flux admits a geometric decomposition. We find that total activity alone can obscure coupling excess: a sampler may approach the minimum activity required by the prescribed marginal transport while still retaining substantial coupling excess. In language generation, as NFE increases, refinement-based samplers accumulate additional trajectory activity with limited diversity gains, whereas MAFTC reduces trajectory activity while maintaining comparable generation quality and higher lexical diversity.

Primary area:
generative models
Keywords:
Discrete Flow Matching, Finite-step Sampling, Activity Geometry, Optimal Coupling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_s7FAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce a concept of activity, which is basically the expected mismatch cost for a coupling distribution on (X,Y). Minimal activity for the cost 1[X\neqY] gives total variation. The concept is thus tied to maximal coupling theory and optimal transport. The authors suggest that as we sample tokens in discrete diffusion, some sampled tokens change in-between sampling iterations and they propose to couple the samples between consecutive time-steps via maximal coupling to reduce the number of these changes. The authors have empirical result that indeed show that the number of changes is smaller compared to the other sampling schemas. However, it is unclear if the proposed strategy leads to any practical benefit.

Critical strengths

Overall paper is well-written and easy to follow. As someone who is familiar with maximal coupling theory, I found the idea of coupling consecutive steps of a sampler, to be very interesting.

I think overall methodology is clear and makes sense.

The empirical results clearly suggest that the number of token changes significantly decreases when using this method.

Critical weaknesses

I have a few questions / concerns about the paper.

First, I am not sure it is appropriate to attribute the proposed coupling in Section 3.3 to be "Minimal-Activity Finite-time Coupling", because it suggests that this coupling is a new contribution of the paper. The technique the authors use is a very standard technique for constructing maximal coupling, see for example [1]. Therefore, I think it is a bit misleading and the authors should clearly cite the works on maximal coupling and say that the technique is a direct application of [1]. I think the real contribution of the paper is the observation that you can apply this technique to consecutive steps of a sampler.

Second, while the technique reduces number of token changes, it does not seem to lead to practical improvements compared to baselines. How can authors explain this? It basically means that reducing activity does not seem to affect quality of sampling. Then, the question is, why should we do this procedure and why should we care about it? While it is simple, it requires doing additional work and computation and therefore increases the computational cost (For example, you need to compute total variation and you need to resample from the adjusted distribution). Is there a regime where this method is particularly well suited?

nit: For readability, in result tables, can you highlight best metrics in bold and second best with underscore? It'll be much easier to read.

At the moment, I'm inclined to reject, but I am open for discussion.

[1] Jacob, P. Lectures on Couplings and Monte Carlo. https://sites.google.com/site/ pierrejacob/cmclectures, 2021.

Questions for authors

See above.

7 comments
AuthorsOpenReview ↗

Response to Reviewer s7FA (No. 43815: The Activity Geometry of Discrete Generative Sampling)

Thank you for your constructive comments and for recognizing the clarity of the methodology and the reduction in token changes. We also appreciate your interest in coupling consecutive sampling steps. Your questions about the relationship to classical maximal coupling and the practical benefit after accounting for implementation cost have helped us clarify the contribution and its computational implications. Below, we respond to these questions and describe the supporting evidence and revisions. We use original manuscript for the version reviewed and revised manuscript for the updated version accompanying this response.

AuthorsOpenReview ↗

W1: Attribution and the contribution beyond classical maximal coupling

We agree that the common-mass/residual construction is classical maximal coupling. MAFTC names the finite-time sampling construction we use between prescribed conditional marginals.

The original manuscript explicitly credits the classical construction: Section 3.1, p. 4, lines 185–192, Eq. (10), introduces the minimum-activity identity with “By the classical maximal-coupling identity,” citing Griffeath (1975) and Thorisson (2000). Appendix A.1, p. 14, titled “Classical maximal coupling and residual pairing,” describes the family of minimizing couplings. Related Work, p. 9, lines 453–461, also acknowledges earlier applications in discrete generative models and explains that MAFTC applies this principle to prescribed conditional marginals.

Thank you also for recommending Jacob's lecture notes, a useful additional reference on coupling methods. Building on the classical result, the original manuscript makes three contributions:

  • Induced versus prescribed transport. Section 3.2, Eqs. (11)–(13), decomposes activity relative to the induced marginal pKpK and separately measures its discrepancy from the prescribed marginal qq. The bound A(K)+ϵ(K)≥TV⁡(p,q)A(K)+\epsilon(K)\geq\operatorname{TV}(p,q) shows why low activity can reflect insufficient marginal transport. Evaluating activity alongside marginal discrepancy distinguishes insufficient transport from reduced coupling excess.
  • One-step versus trajectory activity. Section 3.5, p. 6, lines 288–302, distinguishes coupling excess from excess Hamming path length. A trajectory can contain repeated changes even when each constituent coupling is optimal.
  • Excess activity in kinetic-optimal dynamics. Theorem 4.2, p. 7, Eqs. (26)–(27), identifies exactly which kinetic-optimal flux contributes activity above the minimum on Gibbs paths. Section 5.1 and Appendix E.3 apply this result to FUDOKI: the endpoint-averaged integrated activities are 4.01244.0124 for the kinetic-optimal dynamics and 3.03813.0381 for the minimum, although every nonzero kinetic-optimal jump decreases distance to the endpoint.

Together, these results explain how the coupling choice affects activity, why low activity alone can be misleading, and where excess arises in existing dynamics. The revised manuscript brings these uses into clearer focus in contribution (ii), the transition from Section 3 to Section 4, and the organization of Section 5.

AuthorsOpenReview ↗

W2 (1/2): Practical benefits and computational overhead

We understand the concern as whether changing the coupling provides a useful benefit once generation quality and implementation overhead are considered. Even kernels realizing the same prescribed marginals can produce different numbers of changes. Table 6 in the original manuscript shows that MAFTC's trajectory excess decreases as NFE increases, while the baselines' excess increases. Lower Gen-PPL can come at the cost of reduced diversity, so we compare diversity at similar Gen-PPL. Appendix Figure 3 compares Gen-PPL with corpus coverage and within-sequence repetition on the same 2,048 generated sequences per configuration. In the revised manuscript, the trajectory results appear in Figure 1(d) and Table 7, and the quality–diversity curves appear in Figure 2. Section 5.2 explains these complementary aspects of diversity at similar Gen-PPL, and Appendix E.6 gives the corresponding sequence and pooled entropy analysis.

Reviewer_5EKcOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies discrete diffusion models with a uniform transition kernel and proposes a sampler design based on dynamic optimization.

Critical strengths

The paper provides a fairly in-depth discussion of sampling for this class of models and offers theoretical guidance that could be valuable.

Critical weaknesses
  1. Although the paper falls within the field of generative models, much of the main text is devoted to complex derivations, while intuitive evidence, visualizations, and experimental results are relatively limited. This makes the paper feel unbalanced. Readers should be able to understand the main results without relying on the appendix, so the presentation and organization could be improved.

  2. The evaluations of image and text generation are insufficient. For image generation, practical usefulness depends on several factors, including instruction following, composition, and aesthetic quality. GenEval alone cannot adequately establish the method’s advantages. I suggest adding at least one metric related to aesthetic quality, such as HPSv2 or HPSv3, along with some visual examples. For text generation, a more comprehensive Gen-Entropy versus Gen-PPL curve would help show the overall trend and reduce concerns about selective reporting.

  3. The paper’s overall motivation and ultimate contribution remain unclear. Reducing token change by itself does not appear to make a substantial difference to inference or deployment: the method does not significantly improve inference speed or memory usage, nor does it demonstrate a substantial gain in generation quality. The practical value of optimizing this quantity therefore needs clearer justification.

Questions for authors

My main concerns are covered above.

In particular, could the authors provide more image-generation comparisons and the commonly used Gen-Entropy versus Gen-PPL curves for text generation? These would allow a more intuitive and broader assessment of the sampler’s improvements. Comparisons spanning NFE values from 8 to 1024 are also common in this area.

9 comments
AuthorsOpenReview ↗

Response to Reviewer 5EKc (No. 43815: The Activity Geometry of Discrete Generative Sampling)

Thank you for the time and effort you devoted to reviewing our manuscript and for recognizing the potential value of our theoretical analysis. Your comments on the balance of the presentation, the breadth of the evaluation, and the practical value of reducing activity have helped us clarify the connection between the theory and the experimental evidence. Below, we address these concerns, distinguish the evidence already present from the new analyses and measurements, and explain the corresponding revisions. We use original manuscript for the version reviewed and revised manuscript for the updated version accompanying this response.

AuthorsOpenReview ↗

W1: What does the theory explain, and can its conclusions be understood from the main text?

We agree that readers should be able to understand the main conclusions and their empirical relevance without having to reconstruct the argument from the appendix. The central question is which token changes are required by the intended probability transport and which arise from the choice of coupling. This distinction matters: a sampler can make few changes simply by moving too little probability mass, while an exact process can make extra changes even when every jump approaches the endpoint.

In the original manuscript, Section 3 separates coupling excess from marginal mismatch, Theorem 4.2 identifies excess in kinetic-optimal Gibbs dynamics, and Section 5.1 evaluates both mechanisms. Section 3 treats general categorical marginals, while Section 4 studies the metric-induced Gibbs paths used in FUDOKI. Uniform-source text generation is one application.

Revision: In the revised manuscript, the opening of Section 4 explains why we apply Section 3's minimum-activity reference to existing dynamics: to assess whether following the prescribed path and approaching the endpoint at every jump are sufficient to minimize activity. Section 5 presents the evidence in the same order: activity and marginal-error diagnostics, terminal quality and diversity, and actual execution cost. We have moved the quality–diversity Figure 3 from the original manuscript into the main text as Figure 2, and added the quality–cost comparison in Table 2. The main text now brings this empirical evidence together with the theory that explains what activity measures.

AuthorsOpenReview ↗

W2 (1/2): Could the reported text quality reflect reduced diversity or selective reporting?

An entropy–perplexity curve helps address a real concern: lower Gen-PPL can accompany more repetitive or less diverse output. We therefore evaluate perplexity together with diversity, following the approach in Duo, Section 5.2 and Ψ\Psi-Samplers, Appendix D.1.2, which report Gen-PPL alongside sequence unigram entropy.

The original manuscript already includes quality–diversity curves. Its Appendix Figure 3 plots pooled Dist-4 and average sequence Rep-4 against Gen-PPL for all three samplers at 16, 32, 64, and 128 NFE. Its Table 5 reports both unigram entropy definitions, Dist-1/2/4, Rep-4, and Gen-PPL across all seven budgets from 2 to 128 NFE. For example, at 32 NFE, MAFTC and Ψ\Psi have similar Gen-PPL, 33.5933.59 and 33.5133.51, respectively. MAFTC has higher pooled Dist-4 (0.85150.8515 versus 0.84930.8493), while Ψ\Psi has lower Rep-4 (0.02170.0217 versus 0.02270.0227). Thus corpus coverage and within-sequence repetition favor different samplers. Figure 4 explains the higher coverage through reduced cross-sequence four-gram reuse at this operating point. In the revised manuscript, Section 5.2 and Figure 2 now make this distinction explicit, and Table 6 retains the complete measurements.

The two entropy definitions provide a complementary view. HseqH_{\rm seq} averages unigram entropy computed separately within each sequence; HpoolH_{\rm pool} uses token frequencies pooled across all sequences. Both were defined in Appendix E.1 and reported in Table 5 of the original manuscript. Near Gen-PPL 30, Ψ\Psi has higher HseqH_{\rm seq} but lower HpoolH_{\rm pool} than both other samplers. These rankings are consistent: pooling incorporates differences in token usage across outputs as well as entropy within each output. The per-sequence aggregation corresponds to that used in the Duo/Ψ\Psi evaluation code, with our token-handling convention specified in the manuscript. Unigram entropy ignores token order, so it complements the four-gram coverage and repetition curves.

Revision: In the revised manuscript, Figure 4 plots both entropy definitions against Gen-PPL using the existing measurements over the same 16–128 NFE range as the diversity curves in the original manuscript. Appendix E.6 now explains the different rankings with an exact decomposition for equal-length outputs: pooled entropy equals average sequence entropy plus the average Kullback–Leibler divergence of each sequence's token-frequency distribution from the pooled distribution. Near Gen-PPL 30, this divergence is 2.0052.005, 1.8431.843, and 1.9061.906 nats for Duo+DCD at 128 NFE, Ψ\Psi at 64 NFE, and MAFTC at 64 NFE, respectively. Thus Ψ\Psi's higher sequence entropy is outweighed by smaller differences in token frequencies across outputs. Figure 4 includes uncertainty bars for Gen-PPL and sequence entropy; Table 8(b) summarizes paired differences in quality, activity, entropy, and repetition, with all evaluated confidence intervals in the supplementary archive. Readers can now examine the entropy curves alongside corpus coverage and repetition.

In the revised manuscript, Appendix E.6.2 (Table 9) reports the conditional-coupling interpolation ablation. Appendix E.6.3 documents the completed decoding search and controls with a fixed refresh acceptance probability; all 78 configurations from the main evaluation and 315 pilot configurations are included in the accompanying supplementary_tables.zip. Search coverage differs across methods, and the tables specify the settings and size of each search. The results include settings where additional refresh lowers Gen-PPL while reducing coverage, as well as tuned Ψ\Psi settings with competitive quality. These results help show the trade-offs between quality, diversity, and activity.

Reviewer_B6Z8OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper studies the dynamics of sampling trajectories in discrete diffusion models, in particular the phenomenon of inefficient sampling, with repeated masking and unmasking of the same token. The central object is the activity of the transition kernel learned by the model, i.e., the expected trajectory distance. The paper analytically decomposes this quantity into required and excess activity, facilitating a sampling algorithm (MAFTC) that leads to more "efficient" trajectories in text and image generation tasks.

Critical strengths

The main strength of the submission is the strong theoretical idea and the well-executed presentation and analysis.

S1: Very Well Written Background and Section 3.

The Idea of activity decomposition and the general theory were very pleasant to follow. I think this drastically increases the value of a paper like this. Concepts and symbols were consistently introduced; the background made the notation very clear, etc. I hope the authors maintain this writing style; if a similar style is maintained for the experiments and Section 4, the paper would be a strong contribution.

S2: Very Clean Theoretical Contribution.

The conceptual idea behind decomposing activity into required and excess is very clean and well-analysed. If the experimental results are strengthened, this feels like the discrete version of rectified flows, potentially a useful result.

Critical weaknesses

The weaknesses of this manuscript are that, with Algorithm 1, the presentation becomes much less understandable for me. Because of this, i fear that the sampling algorithm might require access to the target distribution, rendering it practically useless. Aside from Algorithm 1, the theory does not seem to be utilised in this manuscript.

W1: Requirements of the Sampling Algorithm are Unclear.

In line 9, Algorithm 1 samples from [q−p]+/τ[q-p]_+/\tau. I did not see an explanation of this, but if this requires access to qq, this is a huge issue for a sampling algorithm. Given that this sampler is the only way the theoretical contribution is utilised, if it is just a "toy" sampler, the manuscript's contribution would be substantially weakened. Thus, my decision hinges on the authors' response and clarification of this result; the current rating assumes the sampler does not require analytic access to qq.

W2: Purpose and Presentation of Section 4 unclear.

It seems that Section 4 generalises the discrete-time results to continuous-time settings and compares the "kinetic-optimal" solution to the minimal-activity solution. It is not quite clear to me what the purpose of this section is. While the results may be mathematically nice, do they serve a purpose in the manuscript?

W3: Experimental Results have no error bars, seem weak.

The differences in the experimental setting are hard to interpret without quantifying the numerical uncertainty. The only effect that seems stable as currently presented is that the NTC is much lower in text generation, i doubt that the other effects are statistically significant. Please correct me if I am mistaken.

Questions for authors

Q1: MAFTC Sampling Algorithm, Target and Requirements

What is the precise output of the MAFTC algorithm? Line 9 samples from [q−p]+/τ[q-p]_+/\tau. What is meant by this quantity? Does it require access to qq?

Q2: What purpose does Section 4 serve in the manuscript?

Section 4 (and also Section 3.5) appear a bit disjointed in the manuscript. What purpose do they serve?

Q3: MAFTC as training objective.

To my understanding, you do not propose a fine-tuning objective based on MAFTC. However, would that not be possible? And potentially be a strong contribution? It feels like a discrete version of rectified flows.

7 comments
AuthorsOpenReview ↗

Response to Reviewer B6Z8 (No. 43815: The Activity Geometry of Discrete Generative Sampling)

Thank you for your detailed and insightful review. We appreciate your recognition of the activity decomposition in S2 and the clarity of the background and Section 3 in S1. Your suggestion to carry this explanatory style into Section 4 and the experiments has guided our revisions: we now make the motivation for each analysis and the role of its supporting evidence more explicit. Below, we address your concerns and identify the corresponding changes. We use original manuscript for the version reviewed and revised manuscript for the updated version accompanying this response.

AuthorsOpenReview ↗

W1/Q1: Does Algorithm 1 require an unavailable distribution, and what does it return?

We agree that evaluating qq would be a serious problem if it denoted the unknown unconditional distribution at the next time, or the data distribution itself: the sampler would then need access to an unavailable distributional oracle.

Algorithm 1 does evaluate qq, which here is a known conditional path distribution. In the original manuscript, line 4 samples an endpoint token aia^i from the learned predictor, and line 6 sets

q=p_{t_{n+1}}^i(\cdot\mid a^i).$$ These vectors are determined by the specified conditional path, the sampled endpoint, and the time grid. No ground-truth endpoint is supplied during generation. The common-mass/residual construction in Section 3.3 of the *original manuscript* gives the destination distribution in line 9: $$r(y)=\frac{[q(y)-p(y)]_+}{\tau},\qquad \tau=\sum_y [q(y)-p(y)] _+=\operatorname{TV}(p,q).$$ When a move occurs, this distribution selects its destination among states whose conditional probability increases. When $\tau=0$, the state is retained. For the uniform-source text path, this residual destination is the sampled endpoint itself. For the image Gibbs path, the vectors are computed from the known codebook distances and schedule. Algorithm 1 returns $\widehat X_N$, the final token sequence, which is then decoded into text or an image. The pretrained-model experiments use this sampler for direct token replacement on both paths.
AuthorsOpenReview ↗

S1 and W2/Q2: What does Section 4 contribute, and how is the theory used beyond Algorithm 1?

Section 4 shows how Activity Geometry can diagnose existing dynamics. In Section 4.2 of the original manuscript, Theorem 4.2 identifies excess jump activity in the kinetic-optimal flow used on Gibbs paths, even though every nonzero jump decreases distance to the fixed endpoint.

Specifically, for endpoint aa, let

H=\{x:\rho(x,a)>\mu\},\quad L=\{x:\rho(x,a)\le\mu\}.$$ Here $\mu$ is the average endpoint-distance score under the current conditional distribution. The theorem states that the activity above the minimum in the kinetic-optimal flow equals the total probability flux of transitions entirely within $H$ or entirely within $L$. Thus, moving toward the endpoint can still involve more jumps than the prescribed marginal evolution requires. These transitions affect how each state's probability evolves. Simply removing their flux generally changes the path, so reaching the minimum requires reallocating transport. Section 5.1 and Appendix E.3 of the *original manuscript* evaluate this result. Across 32 FUDOKI conditional paths, the endpoint-averaged integrated activities are $4.0124$ for the kinetic-optimal flow and $3.0381$ for the minimum. Their difference, $0.9743$, is precisely the integrated flux identified by the theorem. Separately, Section 5.1 uses the finite-time decomposition to explain why Euler's total activity can be close to the prescribed minimum while still containing substantial coupling excess. Section 3 therefore provides tools both for constructing MAFTC and for diagnosing existing transitions; Section 4 develops this second use. The framework applies to general categorical transitions, including the uniform and metric-induced paths evaluated here. **Revisions:** The opening of Section 4 in the **revised manuscript** explains why we apply Section 3's minimum-activity reference to existing dynamics: to assess whether following the prescribed path and approaching the endpoint at every jump are sufficient to minimize activity. Section 4.1 then makes the connection between the two decompositions explicit. For an exact CTMC following the prescribed path, $\epsilon(P_{t+h\mid t})=0$ and, under the stated local regularity assumptions, $$\lim_{h\downarrow0}\frac{\chi(P_{t+h\mid t})}{h} =\mathcal A_t(j)-\frac12\|\dot p_t\|_1.$$ In the **revised manuscript**, this relation appears as new Eq. (22), proved in Appendix A.5. Eq. (24) moves the finite-interval Gibbs threshold from Appendix B, Eqs. (40)–(41), of the *original manuscript* into the main text and shows how it approaches $\mu$. Section 4 is now titled **“Activity Geometry of Kinetic-Optimal Flows.”** We moved the definitions and explanations of trajectory measurements from Section 3.5, Eq. (18), in the *original manuscript* to the experimental setup, Eq. (28), in the **revised manuscript**. This retains the distinction between trajectory quantities and one-step coupling excess, and explains how averaging trajectory activity over generated samples gives the reported NTC. The experiments now present the existing activity diagnostics in Section 5.1, quality and diversity in Section 5.2, and the added execution study in Section 5.3. Contribution (ii), the background paragraph on sampling, and the conclusion also make this connection clearer. Together, these changes extend the explanatory style you appreciated in S1 to Section 4 and the experiments.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal (No. 43815: The Activity Geometry of Discrete Generative Sampling)

Dear Area Chair and Reviewers,

First of all, we would like to express our sincere gratitude to the Area Chair and reviewers for their time, effort, and recognition given to our manuscript.

It is worth pointing out that the reviewers' comments have helped us improve the manuscript quality. Generally, B6Z8 highlighted the clarity of the activity decomposition, 5EKc noted the potential value of the theoretical guidance, and s7FA recognized the value of designing samplers by choosing how to couple the prescribed conditional marginals at consecutive sampling times.

Below we briefly summarize the main considerations from three reviewers. Unless otherwise indicated, section, figure, and table numbers refer to the revised manuscript.

1. What does Algorithm 1 need at sampling time, and what does it return? (B6Z8 W1/Q1)

In lines 4 and 6 of Algorithm 1 in the original manuscript, the sampler draws an endpoint token from the pretrained predictor at each position and evaluates the prescribed path conditional on that token at two consecutive times to obtain pp and qq. The sampler can therefore evaluate qq without querying the unknown data distribution. Algorithm 1 returns the final token sequence for decoding.

2. What Activity Geometry adds to classical maximal coupling (s7FA W1, B6Z8 W2/Q2 on Section 4)

The original manuscript credits classical maximal coupling. We added Jacob (2021), as suggested by s7FA. Section 3.2 measures coupling excess relative to the minimum activity required to reach the marginal a sampler actually produces, pKpK. If the probability of changing a token falls below the minimum required to reach the prescribed marginal qq from pp, then pKpK must differ from qq.

Theorem 4.2 in the original manuscript shows that kinetic-optimal Gibbs dynamics can follow the prescribed path and still incur excess activity even when every jump approaches the endpoint. We revised the opening of Section 4 to explain this application. New Eq. (22) connects it to Section 3 by showing that one-step coupling excess per unit time converges to excess jump activity.

3. Practical value of fewer token changes and MAFTC’s sampling overhead (5EKc W3, s7FA W2)

Figure 1(d) decomposes the average cumulative number of token changes per sample (NTC) into net displacement and trajectory excess. As the number of function evaluations (NFE) increases from 8 to 128, average trajectory excess for text falls from 142.8 to 54.2 changes per sequence for MAFTC and rises for both Duo+DCD and Ψ\Psi (Table 8). To test whether fewer token changes translate into lower inference cost, we added an end-to-end comparison in Section 5.3. On the uniform-source text path, MAFTC can identify which positions need endpoint predictions before evaluating the vocabulary head. With the same selective execution scheme for all three samplers, MAFTC saves 40.87 ms per request at 64 NFE and 48.38 ms at 128 NFE relative to Duo+DCD on an RTX 4090 at batch size one, including sampling overhead (Table 2 and Appendix E.7).

4. Text quality, diversity, and uncertainty (5EKc W2 and NFE question, B6Z8 W3)

We evaluated seven budgets from 2 to 128 NFE. Within 16–128 NFE, we matched MAFTC to each baseline at three generative perplexity (Gen-PPL) levels. All six pairs differ by less than 1.5% in Gen-PPL (Figure 5).

At comparable Gen-PPL, MAFTC achieves higher corpus-level diversity (pooled Dist-4) and lower NTC in all six comparisons with Duo+DCD and Ψ\Psi (Figure 2 and Tables 7–8). No higher NFE budget is needed to establish this joint advantage.

MAFTC also has higher pooled unigram entropy than Ψ\Psi at all three Gen-PPL levels (Figure 4). In the same comparisons, Ψ\Psi has higher average sequence unigram entropy and lower within-sequence four-gram repetition (Rep-4) than MAFTC.

We moved the original quality–diversity curves into the main text (Figure 2) and used the existing measurements to plot both pooled and average sequence unigram entropy against Gen-PPL (Figure 4). We also extended the paired bootstrap analysis to additional budgets and metrics. Table 9(b) reports 95% confidence intervals for paired differences in Gen-PPL and Rep-4 at the fixed generation seed.

5. Image examples and HPSv3 evaluation (5EKc W2 and follow-up)

We added image examples generated from the same prompts and initial tokens for all three samplers (Figure 3). We also scored all 39,816 images from the original GenEval comparison with HPSv3. At the original guidance scale of 5, HPSv3 and GenEval give MAFTC the same rank by point estimate at each of the six budgets from 4 to 128 NFE (Tables 4 and 6). Appendix E.5.3 reports GenEval, HPSv3, and NTC at CFG=3 for MAFTC, Euler and TC samplers on the same 553 prompts and at the same six NFE budgets. TC's HPSv3 lead over MAFTC narrows at each successive budget, from 0.928 to 0.018 points.

We thank all reviewers and the Area Chair again for their time and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.