acceptodds
Under review as a conference paper at ICLR 2027

LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flow

Abstract

Uniform discrete flow permits repeated updates at every generation position. While continued revision supports correction of wrong tokens, it also exposes correct intermediate predictions to later errors. An experiment on Sudoku puzzles shows that 9.4% of generated cells are correct at an intermediate step but incorrect in the final output. We introduce generation order into uniform discrete flow through selective absorption, which fixes chosen predictions while preserving the uniform-flow velocity at active positions. To prioritize reliable predictions for absorption, we propose Low-Entropy Discrete Flow (LEDFlow), a training-free sampler that adaptively orders absorption by local entropy. By decomposing absorption error into joint dependence and conditional prediction terms, we show that, under entropy-error regularity, selecting the lowest-entropy positions under a fixed absorption count minimizes an upper bound on the conditional term. We further analyze sensitivity of global lookahead, whose worst-case decision-error bound grows with lookahead window under an imperfect denoiser. Across reasoning benchmarks, LEDFlow attains 0.845 Nikoli Sudoku solve accuracy, with largest gains on strongly constrained tasks. On a text-to-image generation benchmark it attains the best overall score among decode-time samplers, and on multimodal understanding it improves over the default sampler on all six benchmarks, at an inference cost comparable to standard flow sampling.

Primary area:
generative models
Keywords:
uniform discrete flow matching, generative models, selective absorption, multimodal generation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.40 over 5 reviews, above 68% of the others.
Reviewer_sqK1OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LEDFlow, a training free sampler, which absorbs lowest entropy positions in discrete flow through selective absorption.

Critical strengths

Emperical analysis. The emperical analysis is good. Multiple samplers are compared on a range of tasks (multimodal udnerstanding, text-to-image generation, constrained reasoning) and multiple datasets. The proposed approach consistently performs well, achieving top ranks across the board.

Critical weaknesses

Theory

Definition 1: This is a very key point so the argument should be strengthened. Is this a realistic assumption to make? Additional validation should be performed or clearer justification on why this is the case.

Presentation The presentation could be improved and simplified. Some more minor suggestions to improve clarity:

  • Global lookahead/info gain: This takes up a significant amount of room/discussion, e.g. at the end of Section 4. As this is just a comparison, it would be clearer to move such discussions to the main body. It can detract from the main focus, and the additional discussion e.g. in line 85-87 feels misplaced.

  • In the abstract, mentioning 0.845 solve accuracy is not very useful without closest comparison mentioned. Either remove it or add other results for comaprison.

  • L155-161: Everything relies crucially on the denoiser itself. Please promote anything important from the appendix to the main body to strengthen these claims, or add some experimental validation around the denoiser itself, e.g. how entropy correlates with correctness empirically.

  • Figure 2 caption says LEDflow rises monotonically. This just happens to be the case in one instance? Can error bars be added to this plot, or multiple lines per run.

  • In figure 1 it looks like entropy is normalised 0..1, but this isnt the case in the paper (L194). A note should be added to the fig to avoid confusion.

  • Algorithm 1 could benefit from adding in line natural language comments, or replacing some of the heavy notation.

  • The appendix is very difficult to parse in its current structure.

Questions for authors

The main point to clarity is around Theorem 1 (and Definition 1), this entropy-error regularity. How often is this idealized case violated? Its mentioned 95.5% of sampled Sudoku states? What is the impact when it's violated, e.g. in the 4.5% of cases? The paper crucially relies on this, so more analysis is warranted.

  • In the results tables -- Can you please confirm how the rank is generated, are the scores averaged over 5 runs, and then the rank based on these averages? Or is the rank computed for each run and averaged?

  • In figure 2, its clear that earlier stopping could help the uniform flow. Can early stopping criteria be compared to as well, e.g. adding conditions exploring just earlier stopping rather than selective absorption.

5 comments
AuthorsOpenReview ↗

Replies to Reviewer sqK1: Weaknesses

Thank you for your comments. We have updated the paper with a new PDF file uploaded with your comments accordingly (updates in blue). Please find our replies as follows:

Weaknesses:

Theory

Def. 1:

We provide the following example using Sudoku for its unique trajectory solution. The ϕ\phi-regularity follows the relationship between predictive entropy and probability assigned to the correct digit. Given state ss where previous absorbed positions are correct, at each remaining position jj, the true conditional distribution should assign probability one to the solution digit yj∗y_{j}^{\ast}, so that the KL divergence from the denoiser's predicted distribution Qθ(⋅∣s,j)Q_{\theta}(\cdot\mid s,j) is −log⁡Qj(yj∗)-\log Q_{j}(y_{j}^{\ast}). Now let pjmax⁡=max⁡zQj(z)p_{j}^{\max}=\max_{z} Q_{j}(z) and define log probability ratio Δj(s)=log⁡(pjmax⁡/Qj(yj∗))\Delta_{j}(s)=\log(p_{j}^{\max}/Q_{j}(y_{j}^{\ast})), we have H(Qj)=EZ∼Qj(−log⁡Qj(Z))H(Q_{j})=\mathbb{E}_{Z\sim Q_{j}}(-\log Q_{j}(Z)), which is greater than or equal to −log⁡pjmax⁡-\log p_{j}^{\max}, so that Dj(s)=−log⁡pjmax⁡+Δj(s)≤H(Qj)+Δj(s)D_{j}(s)=-\log p_{j}^{\max}+\Delta_{j}(s)\leq H(Q_{j})+\Delta_{j}(s). When the correct digit has the highest predicted probability, the ratio is zero so the predictive entropy bounds KL divergence. If the ratio is less than cH(Qj)cH(Q_{j}) for some non-negative constant cc, ϕ(h)=(1+c)h\phi(h)=(1+c)h then satisfies Def. 1. Next, regarding empirical evidence, we show that Def. 1 satisfies 95.5% of sampled reachable states for the model we use in main text, where a relaxed envelope in Eq. 50: KL⁡(p1∣skq(a1j)∥Q1∣tkθ(a1j∣sk))≤ϕ0+ϕ(Hθj(sk))\operatorname{KL}\left(p^{q}_{1\mid s_{k}}(a_{1}^{j})\middle\Vert \mathbb{Q}^{\theta}_{1\mid t_{k}}(a_{1}^{j}\mid s_{k})\right)\leq \phi_{0}+\phi\left(H_{\theta}^{j}(s_{k})\right), where ϕ0≥0\phi_{0}\geq 0, further holds on 99.4% of the states. Table 19 (attached below) also provides details that shows error on each cell rises monotonically with entropy (now App. F.3, Table 19 in the revised manuscript).

HθH_{\theta} (bin mean)0.000.240.460.640.791.08
Per-cell error0.0150.1200.2220.3430.3990.526

Nevertheless, we conducted a further out-of-distribution scenario for Def. 1 by fitting a monotone envelope on the puzzles, which we found the envelope covers 95% trajectories and 99.9% of positions. For selected cells with entropy less than 0.05, at most 0.41% are wrong (App. F.3, Table 17).

Presentation

W1: Global lookahead/info gain:

Our primary contribution introduces ordering in uniform discrete flow via absorbing state, with a stepwise greedy decoding on entropy. There are two questions: 1) Why entropy (Thm. 1) and 2) Why greedy decoding because a recent work [1] proposes that "selecting locally certain tokens ... leads to suboptimal generation", motivating our choice of Top K-margin and Info-Gain as SOTA baselines of each angle. In particular, LEDFlow ranks positions using the current prediction. Lookahead temporarily fixes each candidate to its most likely token, reruns the denoiser, and scores how uncertain the remaining positions would be. We added one line (L316 - 318) to explain why is global lookahead a key idea that we have to analyse and compare with.

0.845 mention:

We add the 0.410 from ablation baseline into both abstract (L24 - 25) and introduction (L87 - 88) to ease comparison.

Justification of entropy and correctness:

Line 287 already cites [1] regarding negative relationship between prediction entropy and reasoning accuracy, which we further verify with Table 19 at App. F.3 (attached below):

HθH_{\theta} (bin mean)0.000.240.460.640.791.08
Per-cell error0.0150.1200.2220.3430.3990.526

As per your suggestion, we add the line with citation [1] at L181 - 182 to improve clarity.

Figure 2 results:

It is run on five seeds (0 - 4) to prevent bias from single instance. We add a line (L172) to clarify it clearly in main text with reference pointer towards Appendix F.1, as well as error bands on Figure 2(a).

Figure 1 note:

We add a note at Figure 1 caption (L67) to state explicitly that the entropy values visualised there are normalised to ease reading.

Algo 1 notation:

Well noted with many thanks. We incorporate some natural language descriptions in Algo 1.

Appendix structure and wordings:

We have added a new page of content for the Appendix and updated wordings in Appendix accordingly to ease reading.

AuthorsOpenReview ↗

Replies to Reviewer sqK1: Questions

Questions:

Q1 Violation of idealised case:

We have detailed theoretical justifications in Theory Weakness Def. 1 above. Regarding the empirical evidence, we show that Def. 1 satisfies 95.5% of sampled reachable states for the model we use in main text, where a relaxed envelope in Eq. 50: KL⁡(p1∣skq(a1j)∥Q1∣tkθ(a1j∣sk))≤ϕ0+ϕ(Hθj(sk))\operatorname{KL}\left(p^{q}_{1\mid s_{k}}(a_{1}^{j})\middle\Vert \mathbb{Q}^{\theta}_{1\mid t_{k}}(a_{1}^{j}\mid s_{k})\right)\leq \phi_{0}+\phi\left(H_{\theta}^{j}(s_{k})\right), where ϕ0≥0\phi_{0}\geq 0, further holds on 99.4% of the states. Table 19 (attached below) also provides details that shows error on each cell rises monotonically with entropy (now App. F.3, Table 19 in the revised manuscript), and cells that violate the bound are wrong.

HθH_{\theta} (bin mean)0.000.240.460.640.791.08
Per-cell error0.0150.1200.2220.3430.3990.526

Q2 Rank generation:

We compute the scores of each run from five seeds and aggregate the scores on each dataset. We then rank those aggregate scores and then compute the average ranking.

Q3: Early stopping:

The absorbing state can somehow be considered as early stopping at each position. Nevertheless, we interpret your idea of early stopping as early stopping over the entire puzzle. On that regard, we pretrain another two checkpoints which 1) replaces the position-wise absorbing indicator with a single early stopping indicator, and 2) adds an additional early stopping indicator by training an additional sigmoid layer to take in the position-wise absorbing indicator. We undergo the same evaluation on Nikoli dataset (App. G.1, Table 22):

ProcedureAccuracy
Uniform flow0.410
Single early stopping indicator0.364
Early stopping on position indicator0.452
Entropy selective absorption0.845

It shows that 1) the position wise indicators contain signals to validate the model's internal thoughts on each position, which is consistent with our comparison at Table 14 at App. F.1 between pretraining without indicator versus pretraining with indicator but not using it at inference, and 2) the current early stopping decisions seem to be overconfident in both designs, so that in training, it loses the learning on trajectories where the low-entropy positions can be refined at later steps. We are happy to discuss further if you have further early stopping criteria which you would like to test.

Please let us know if you have further concerns!

References

[1] Yang et al. Improving sampling for masked diffusion models via information gain. ICML, 2026.

If you have any further concerns regarding our replies, please let us know. We are happy to discuss further in the remaining three days of rebuttal period. Thanks!

Reviewer_jURqOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper targets late-stage churn in uniform discrete flow (FUDOKI), where tokens that are already correct get overwritten near the end of sampling. Its fix adds an irreversible absorption indicator to each position (Eq. 2) and, at each step, fixes the lowest-entropy positions to their argmax. The theory shows that, assuming $KL \le \varphi \left ( H \right ) ,choosingthelowest−entropypositionsminimizesanupperboundontheconditionalerror.HeadlineresultsareNikoliSudoku0.410, choosing the lowest-entropy positions minimizes an upper bound on the conditional error. Headline results are Nikoli Sudoku 0.410\to$0.845 and GenEval 0.753$\to$0.781.

Critical strengths
  1. The controlled ordering ladder is solid: entropy > arbitrary > high-entropy (Table 22). In the n=1000 paired test, entropy order reaches 0.922 vs. Euler's 0.436 (Table 20).

  2. High transparency: I recomputed every rank in Tables 1–3 and all match exactly.

Critical weaknesses
  1. The core numbers are inconsistent. All of the following use the same configuration (Nikoli, 6.4M, K=64):
  • Non-absorbing uniform flow: 0.410 in Table 3 and Fig. 2, but 0.000 in Table 16.
  • Arbitrary-order absorption: three versions, 0.410, 0.380 and 0.340.
  • Single-position LEDFlow: 0.845 vs. 0.900 at the same NFE.

How the "41%$\to$84.5%" gain and the 9.4% motivation figure should be attributed depends on which set is correct.

  1. The large gains may come from the training distribution rather than the sampler. During training, the puzzle denoiser sees a fraction t of positions marked absorbed and filled with ground truth. The non-absorbing baselines keep f≡0f\equiv 0 throughout, so they drift further from the training distribution as t$\to$1. On frozen FUDOKI, where this mismatch does not exist, gains are modest: only POPE is significant among the six understanding benchmarks, and GenEval improves by just +0.028.
Questions for authors
  1. State the protocol behind each Nikoli number and provide one reconciled table.
  2. Re-run the non-absorbing baselines with a denoiser whose training distribution matches inference.
3 comments
AuthorsOpenReview ↗

Replies to Reviewer jURq: Weakness 1 and Question 1

Thank you for your comments. We have updated the paper with a new PDF file uploaded incorporating your comments accordingly (in blue), and have our replies as follows:

Weakness 1:

0.410 vs 0.000:

These are different checkpoints to separate two scenarios, i.e. a uniform flow model pretrained without the indicator and one with the indicator but does not use it for inference. Specifically, the second is done by perturbing the indicator to zero so that the velocity is always non-zero, resulting in resampling at each step. The 0.410 belongs to the second while the 0.000 belongs to the first. We intended to use the second checkpoint in main text to ablate the value of indicator under the same checkpoint, but we provide the first checkpoint result for comprehensiveness (App. F.1, Table 14; App. G.2, Table 25).

0.410 vs 0.380 vs 0.340

The 0.410 refers to above checkpoint with no absorption at inference. The 0.340 refers to the same checkpoint which we keep everything consistent, but let the model predict absorbing without rule-based objective instead of basing on entropy. This serves as additional experiment to understand pretrained behaviour when rule-based objective is not enforced. The 0.380 is an unintentional typo that should be 0.340, which we backtracked in the log and have corrected accordingly. Nevertheless, this does not impact the key comparison between non-absorbing 0.410 and absorbing 0.845 (App. F.1, Table 14; App. G.2, Table 25).

0.845 vs 0.900

The 0.900 refers to a control experiment we incorporate into masked flow scenarios to complement our uniform flow base case, which shows that the benefits of ordering on entropy also generalises to masked scenarios. That serves as an additional experiment we conduct for completeness in Appendix, which we update the paper accordingly to separate that from the 0.845 uniform flow the main paper focuses on (App. G.2, Tables 25 - 27).

Question 1:

Reconciled Table:

There are in total three checkpoints: a uniform flow checkpoint pretrained without absorption indicator (A), one with indicator (B) and one masked flow checkpoint pretrained with indicator (C) (App. G.2, Table 25)

ModelAccuracy
A0.000
B with indicator unused in inference0.410
B with indicator with decision rule not enforced0.340
B with decision rule that picks highest entropy0.110
B with decision rule that picks lowest entropy0.845
C with indicator with decision rule not enforced0.408
C with decision rule that picks highest entropy0.186
C with decision rule that picks lowest entropy0.900
AuthorsOpenReview ↗

Replies to Reviewer jURq: Weakness 2 and Question 2

Weakness 2:

Gains may come from the training distribution

We ran an experiment with the original FUDOKI checkpoint with additional post-training on generated Sudoku puzzles as references. We adopt from dFlowGRPO [1] as GRPO is not directly implementable in discrete flow models (App. G.5, Table 29):

ModelAccuracy
FUDOKI0.000
+ SFT0.395
+ SFT + GRPO0.410

This shows that Sudoku is out-of-distribution for FUDOKI, so changing the sampler would not show performance difference. This motivates our use of a pretrained model with non-zero performance to allow comparison within samplers. Specifically, the 0.410 to 0.845 comparison is conducted on the same checkpoint, with only the difference within the absorbing indicator while keeping everything else consistent, which already takes into account your concerns on training distribution.

Aside from Sudoku, the improvement in image generation is more significant on tasks with more constraints, e.g. Two, Count and Position, whereas absorbing certain areas allows the remaining positions to condition on, along with the prompts. This is consistent with some visualizations we showed in Figure 3(a), whereas a non-absorbing baseline generates three giraffes while the prompt asks for four giraffes. Such absorbing state is beneficial at constrained generation tasks, where the low-constraint subtasks in existing image generation and visual question answering tasks show saturation. We provide following breakdown on generated text length in different VQA subtasks (App. E.1, Table 6).

BenchmarkItemsMean ground truth lengthMean FUDOKI responseMean LEDFlow response
POPE5,1271.001.001.00
MME2,3741.001.001.00
MMBench4,3771.0014.2114.15
GQA12,5781.061.051.05
MMMU1,0501.0052.6452.65
MM-Vet21857.2753.0750.78

While average ground truth word count is small, MMBench and MMMU do not explicitly ask for a single word answer. The model's explanation generated show to benefit the final answer more in the refining process of uniform flow. In contrast, POPE, MME and GQA explicitly ask for a single word answer, so the effect of ordering is not as explicit, as there is no ordering within single token.

Question 2:

Re-run non-absorbing baselines

We train another non-absorbing baseline where the entropy decision rule is not enforced in training so that training distribution matches inference distribution (App. G.1, Table 23):

ModelNikoli performance
Original checkpoint with unused indicator in inference0.410
New checkpoint trained without entropy decision rule0.370

This shows that our initial comparison between 0.410 and 0.845 is not confounded by additional benefits on training distribution mismatch.

Please let us know if you have further concerns!

[1] Wan et al. dFlowGRPO: Rate-aware policy optimization for discrete flow models. Preprint, 2026.

If you have any further concerns regarding our replies, please let us know and we are happy to discuss further. Thanks!

Reviewer_8yJsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This work proposes a variant of uniform discrete flow that uses low-entropy-guided absorption to prevent correct intermediate predictions from being overwritten by incorrect updates. Numerical experiments show improvements across several multimodal understanding benchmarks and substantially higher solve accuracy on Nikoli Sudoku puzzles. The theoretical analysis decomposes local absorption error and shows that, under a ϕ\phi-regularity assumption and for a fixed absorption count, selecting the lowest-entropy positions minimizes an upper bound on the local conditional prediction error.

Critical strengths
  • S1. The work provides a clear motivation for selective absorption by identifying how the divergence of the schedule derivative β˙t\dot{\beta}_t can cause uniform discrete flow to overwrite correct intermediate predictions (Proposition 1). The proposed mechanism is intuitive and easy to implement. Using low predictive entropy as a proxy for reliability offers a novel way to select predictions for absorption, protecting them from potentially harmful subsequent updates.

  • S2. LEDFlow substantially improves the solve rate on Nikoli Sudoku puzzles, from 41.0% for uniform flow without absorption to 84.5%. This 43.5-percentage-point gain provides strong empirical evidence for the effectiveness of selective absorption.

Critical weaknesses
  • W1. The main weakness is the limited justification for the assumptions underlying the theoretical analysis. For a fixed absorption count, the main result shows that selecting the lowest-entropy positions minimizes an upper bound on the local conditional prediction error, provided that the denoiser satisfies the ϕ\phi-regularity assumption. However, this assumption directly imposes the entropy–error relationship needed to justify the selection rule. It remains unclear what theoretical arguments or empirical evidence establish that a meaningful, sufficiently tight bound holds for the models considered.

  • W2. Although LEDFlow substantially improves the solve rate on Nikoli Sudoku puzzles (Figure 2(c)), the gains in the other experiments appear relatively modest (Tables 1–3). The empirical evidence therefore seems strongest for Sudoku, while the broader benefits across tasks are less pronounced.

Questions for authors
  • Q1. What theoretical justification or empirical evidence supports the ϕ\phi-regularity assumption in Theorem 1? In particular, how can we establish that a meaningful, sufficiently tight function ϕ\phi exists for the denoisers considered?

  • Q2. What motivates the choice of the cosine schedule for mkm_k in line 1263? More generally, how should the absorption count be chosen at each sampling step, and how sensitive is performance to this choice?

9 comments
AuthorsOpenReview ↗

Replies to Reviewer 8uJs: Weakness 1/Question 1

Thank you for your comments. We have updated the paper with a new PDF file uploaded, and added the new requested analysis into the paper (updated in blue), with our replies as follows:

W1 / Q1:

We provide the following example using Sudoku for its unique trajectory solution. The ϕ\phi-regularity follows the relationship between predictive entropy and probability assigned to the correct digit. Given state ss where previous absorbed positions are correct, at each remaining position jj, the true conditional distribution should assign probability one to the solution digit yj∗y_{j}^{\ast}, so that the KL divergence from the denoiser's predicted distribution Qθ(⋅∣s,j)Q_{\theta}(\cdot\mid s,j) is −log⁡Qj(yj∗)-\log Q_{j}(y_{j}^{\ast}). Now let pjmax⁡=max⁡zQj(z)p_{j}^{\max}=\max_{z} Q_{j}(z) and define log probability ratio Δj(s)=log⁡(pjmax⁡/Qj(yj∗))\Delta_{j}(s)=\log(p_{j}^{\max}/Q_{j}(y_{j}^{\ast})), we have H(Qj)=EZ∼Qj(−log⁡Qj(Z))H(Q_{j})=\mathbb{E}_{Z\sim Q_{j}}(-\log Q_{j}(Z)), which is greater than or equal to −log⁡pjmax⁡-\log p_{j}^{\max}, so that Dj(s)=−log⁡pjmax⁡+Δj(s)≤H(Qj)+Δj(s)D_{j}(s)=-\log p_{j}^{\max}+\Delta_{j}(s)\leq H(Q_{j})+\Delta_{j}(s). When the correct digit has the highest predicted probability, the ratio is zero so the predictive entropy bounds KL divergence. If the ratio is less than cH(Qj)cH(Q_{j}) for some non-negative constant cc, ϕ(h)=(1+c)h\phi(h)=(1+c)h then satisfies Def. 1. Next, regarding empirical evidence, we show that Def. 1 satisfies 95.5% of sampled reachable states for the model we use in main text, where a relaxed envelope in Eq. 50: KL⁡(p1∣skq(a1j)∥Q1∣tkθ(a1j∣sk))≤ϕ0+ϕ(Hθj(sk))\operatorname{KL}\left(p^{q}_{1\mid s_{k}}(a_{1}^{j})\middle\Vert \mathbb{Q}^{\theta}_{1\mid t_{k}}(a_{1}^{j}\mid s_{k})\right)\leq \phi_{0}+\phi\left(H_{\theta}^{j}(s_{k})\right), where ϕ0≥0\phi_{0}\geq 0, further holds on 99.4% of the states. Table 19 (attached below) also provides details that shows error on each cell rises monotonically with entropy (App. F.3, Table 19).

HθH_{\theta} (bin mean)0.000.240.460.640.791.08
Per-cell error0.0150.1200.2220.3430.3990.526

Nevertheless, we conducted a further out-of-distribution scenario for Def. 1 by fitting a monotone envelope on the puzzles, which we found the envelope covers 95% trajectories and 99.9% of positions. For selected cells with entropy less than 0.05, at most 0.41% are wrong (App. F.3, Table 17).

AuthorsOpenReview ↗

Replies to Reviewer 8uJs: Weakness 2 and Question 2

W2:

Aside from Sudoku, we show that the improvement in image generation is more significant on tasks with more constraints, e.g. Two, Count and Position, whereas absorbing certain areas allows the remaining positions to condition on, along with the prompts. This is consistent with some visualizations we showed in Figure 3(a), whereas a non-absorbing baseline generates three giraffes while the prompt asks for four giraffes. Therefore, such absorbing state is beneficial at constrained generation tasks, where the low-constraint subtasks in existing image generation and visual question answering tasks show saturation. Specifically, we provide the following breakdown on generated text length in different subtasks of VQA (App. E.1, Table 6).

BenchmarkItemsMean ground truth lengthMean FUDOKI responseMean LEDFlow response
POPE5,1271.001.001.00
MME2,3741.001.001.00
MMBench4,3771.0014.2114.15
GQA12,5781.061.051.05
MMMU1,0501.0052.6452.65
MM-Vet21857.2753.0750.78

While the average ground truth word count is small, MMBench and MMMU do not explicitly ask for a single word answer. The model's explanation generated show to benefit the final answer more in the refining process of uniform flow. In contrast, POPE, MME and GQA explicitly ask for a single word answer, so the effect of ordering is not as explicit, as there is no ordering within single token. Figure 4(b) showcase the transferability of entropy ordering to another pretrained backbone, i.e. LLaDA-8B, Dream-7B and MMaDA-8B, which retains the original masked sampler and checkpoints (attached as follows) (App. E.4.2, Table 11):

ModelGSM8KBBHTrackARCCSQA
LLaDA-8B+4.5+6.2+12+2.7−0.4
Dream-7B+12+18+4.8+7.0+5.0
MMaDA-8B+23+0.3+6.0+3.4+3.3

We further attach Table 11 information from Appendix E.4.2 which includes the details of Figure 4(b) for your reference (App. E.4.2, Table 11 in the revised manuscript).

ModelBenchmarkNet LEDFlow improvement over default sampling
LLaDA-8B-Instruct [22]GSM8K+0.045
Dream-7BGSM8K+0.121
MMaDA-8BGSM8K+0.228
MMaDA-8BBBH logical deduction+0.003
MMaDA-8BTracking shuffled objects+0.060
MMaDA-8BARC-Challenge+0.033
MMaDA-8BCommonsenseQA+0.033
LLaDA-8B-InstructBBH logical deduction+0.063
LLaDA-8B-InstructTracking shuffled objects+0.116
LLaDA-8B-InstructARC-Challenge+0.027
LLaDA-8B-InstructCommonsenseQA−0.004
Dream-7BBBH logical deduction+0.176
Dream-7BTracking shuffled objects+0.048
Dream-7BARC-Challenge+0.070
Dream-7BCommonsenseQA+0.050

Nevertheless, we further include the following Dream-7B performance in GSM8K on different denoising steps, which shows that the benefit of entropy ordering also transfers to different denoising steps (App. G.8, Table 33).

Denoiser evaluationsDefault orderEntropy orderDifference
640.3650.436+0.071
1280.3710.492+0.121

Q2:

We use cosine scheduler to align with standard masked diffusion work for comparison [1], with recent work showing its optimality [2]. Nevertheless, we run the following experiment on the Sudoku datasets to compare between different schedulers, which shows that cosine similarity indeed outperforms standard linear scheduler and a control sine scheduler which resonates with the result in [2]. We have added this result into the Appendix as additional hyperparameter sensitivity ablation (App. G.7, Table 32).

Count scheduleRefresh every step
Cosine0.832
Linear0.820
Sine0.816

If you have any other concerns, please let us know and we are happy to discuss further!

Reference

[1] Yang et al. Improving sampling for masked diffusion models via information gain. ICML, 2026.

[2] Zhang et al. The cosine schedule is Fisher-Rao-optimal for masked discrete diffusion models. Preprint, 2025.

If you have any further concerns regarding our replies, please let us know. We are happy to discuss further in the remaining three days of rebuttal period. Thanks!

Reviewer_vwBoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
3(+1)
Read review
Paper summary

It addresses an important problem in discrete flow/diffusion generation: the need to generate multiple correct tokens. But not all may be correctly generated. In the course of generating correct tokens, prior works risk incorrectly replacing previously correctly generated tokens, with no mechanism to undo such changes. This paper provides a methodology, grounded in theoretical foundations, to avert such mishaps, along with experiments on multimodal understanding, text to image generation, and math.

Critical strengths
  • The problem identification has been done well, in both practical relevance and theoretical framing.

  • The overall methodology of modifying the velocity in terms of positions to be fixed and then using entropy to freeze certain predictions over others seems promising.

Critical weaknesses

Please find the weaknesses arranged in three different sections (indicated in bold). Within each section, I have tried to arrange the points to address in decreasing order of importance, so I would encourage the authors to prioritize addressing the weaknesses mentioned in each section of the review accordingly.

The main overall concern with this paper is the severe lack of clarity in writing, and as such, it would be best if the authors could rewrite the whole paper in a much more human-readable way. Perhaps the authors could reduce the overall text as well, by retaining only the pertinent points. Specific comments follow. Many more points on presentation can be raised if one takes a more detailed look at the Appendix (after about half through the Appendix, I had to give up due to other commitments).

  • Technical

I don't understand how exactly global lookahead is used as a baseline, and I didn't go through the math either very well; please explain it clearly. [Major]

The reasoning in lines 1034-1036 is not clear. What is the total number of sampling steps needed when residual uncertainty persists? Please be more mathematical about this. [Minor]

  • Presentation

In L123, the conditioning for Q1∣t(⋅∣at,Ct)\mathbb{Q}_{1 \mid t}(\cdot | a_t, \mathcal{C}_t) is defined as the pair at,Cta_t, \mathcal{C}_t, but this notation is changed later in L192, where the conditioning is sks_k. Please correct this mismatch by defining the notation properly by clearly denoting what is the random variable for this denoiser posterior, what are the values it can take and so on. [Major]

Related to the above, I am having difficulty understanding what the random variable is and what is fixed in Eqn. 4. If a1ja_1^j is fixed, then what's the point of taking an argmax over S\mathcal{S} via zjz_j? [Major]

Please rewrite the paragraph on L1286 in clear words. [Major]

Please use a better word than "frozen-rate jump probability" in L148 [Minor].

  • Experiments

In L1074, "hazard" is not clearly defined, so I don't understand what exactly Figure 5 is measuring. As has already been indicated many times in the review, please explain it in simple and clear terms using math; if needed, use AI tools only for refining words, not creating entire sentences. [Major]

In L1185-1186, given equation 2, how is it possible to use only the indicator and not do any absorption, and yet observe any improvement? My impression was that once something can be absorbed, only then is the corresponding f set to 1, which changes the effective velocity. [Major]

Questions for authors

Q1. In line 079 of the paper, what does "Absorbing several positions independently can discard their dependence," exactly mean? Does it mean that the dependence among positions is removed so that they become independent of each other?

Q2. In L82-83, please clarify what "oracle-reachable" means. I am aware that there is some description of this on the next page, and I probably understand the intended intuition (states that can be reached via the given process, assuming the given data distribution, right?), but this is not a very intuitive term. Better to explicitly describe what is meant by this in the introduction itself in human-readable terms than to have an AI tool give a creative word for it.

Q3. In L040, what is "schedule gradient"? I understand it from the math, but it would be better to define it in a self-contained way before readers have to get to the math later.

Q4. In L176, if the absorption schedule can be fixed in advance, that means both the absorption counts and the positions to absorb at each time step can be fixed in advance. So what does it mean that the "order" can differ then? If I understood the absorption schedule idea wrongly, please reword it in simple words and explain it.

Q5. In L197, how is mkm_k chosen? In the appendix, it says it uses a cosine schedule formula, but there is no discussion of the intuition behind why it should work.

Q6. In L857, what does "pair" refer to in "Discrete flow matching specifies a conditional path between each coupled pair and marginalizes the pair"?

Q7. In Eqn. 17 in the Appendix, I don't see a0a_0 playing a role in the term inside the expectation, so why do we need to marginalize over it?

Q8. L979 reads, "The transition is not merely permitted but is the one the velocity most favours, because a^1j\hat{a}^j_1 is the closest state to the value the velocity is conditioned on." However, this does not seem to be based on sound reasoning: just because it is the closest state and it is permissible, it is not clear why the velocity should "favour" it the most. Please define "favour" here more mathematically. For instance, it might have made more sense to define favour in terms of "utj(zj,at∣a^1j)u^j_t(z_j, a_t \mid \hat{a}^j_1)." If so, the statement here doesn't seem necessarily true. Please clarify this.

Q9. How is the statement on "jump probability" true? Also, please clearly define what "jump probability" is. I would also highly recommend using more standard terminology prevalent in the literature; these seem more like AI-generated words. While that alone may be acceptable under the terms and policies of this conference, using terms that may not clearly convey the underlying intuition is usually not healthy practice in this community.

20 comments
AuthorsOpenReview ↗

Replies to Reviewer vwBo: Technical Weaknesses 1 - 2

Thank you for your comments. We truly appreciate your detailed review over our work. We have updated the paper with a new PDF file uploaded and incorporated your comments accordingly in the paper (updates in blue). **Based on your suggestions, we incorporate additional updates with more standard terminologies. ** Our replies are as follows:

Weaknesses

Technical

W1: Global lookahead:

Our primary contribution introduces ordering in uniform discrete flow via absorbing state, with a stepwise greedy decoding on entropy. There are two questions: 1) Why entropy (Thm. 1) and 2) Why greedy decoding because a recent work [1] proposes that "selecting locally certain tokens ... leads to suboptimal generation", motivating our choice of Top K-margin and Info-Gain as SOTA baselines of each angle. In particular, LEDFlow ranks positions using the current prediction. Lookahead temporarily fixes each candidate to its most likely token, reruns the denoiser, and scores how uncertain the remaining positions would be. We added one line (L314 - 316) to explain why is global lookahead a key idea that we have to analyse and compare with.

Regarding the math, the original Info-Gain Sampler [1] is designed for masked diffusion models and selects unmasking action aa by estimating the entropy drop in unmasking the token ss instead of another hypothetical s′s': JIG(a∣s)=H(s)−H(s′)−Immediate uncertainty costJ_{\text{IG}}(a\mid s)=\mathcal{H}(s)-\mathcal{H}(s')-\text{Immediate uncertainty cost}, evaluated using denoiser predictions on different hypothetical next states. We adapt this formulation into uniform discrete flow by replacing masked setting with selective absorption. For each unabsorbed position jj, we hypothetically absorb its most likely predicted token z^j\hat{z}_{j} for the resulting state sj,z^js^{j,\hat{z}_{j}}, to compute the objective that penalizes the uncertainty of candidate position jj and average entropy of remaining m−1m-1 positions after hypothetically absorbing jj: Jj(s)=−Hθj(s)−1m−1∑i≠jHθi(sj,z^j)J_{j}(s)=-H_{\theta}^{j}(s)-\frac{1}{m-1}\sum_{i\neq j}H_{\theta}^{i}(s^{j,\hat{z}_{j}}). Essentially we use the exact same cosine scheduler so that the number of positions to be absorbed each step is consistent, just that the positions to be considered are different due to policy difference. We add this detailed formulation of original Info-Gain into App. D.3.

Under this formulation, our Lemma analyses how errors in prediction impact the global lookahead ordering. By comparing two candidate positions jj and j′j', the entropy term of current state cancels out because they are shared by the candidates. However, the uncertainty on remaining (m−1)(m-1) positions differs as these remaining positions are conditional on the candidate. Therefore, by denoting δ\delta the error bound between predictive model and oracle entropies, and ϵ\epsilon the bound of total variation distance between token distributions of predictive model and oracle, Lemma 1 shows the worst-case bound on difference between score difference estimated by model and oracle: ∣(Gθj(s)−Gθj′(s))−(Gqj(s)−Gqj′(s))∣≤2ηm\left\lvert\left(G_{\theta}^{j}(s)-G_{\theta}^{j'}(s)\right)-\left(G_{q}^{j}(s)-G_{q}^{j'}(s)\right)\right\rvert\leq 2\eta_{m}, where ηm=(m−1)(δ+ϵlog⁡∣S∣)\eta_{m}=(m-1)\left(\delta+\epsilon\log\lvert\mathcal{S}\rvert\right), with GθG_{\theta} and GqG_{q} referring to information gain estimated by model and oracle. This bound increases with the number of aggregated positions and the denoiser's estimation errors. In contrast, greedy decoding on predictive entropy per steps only results in a bound of 2δ2\delta, so that such global lookahead is more sensitive towards imperfect denoiser predictions. Our empirical analysis in Figure 4(c) supports this motivation, showing that score-estimation error increases with the lookahead window, while the ordering accuracy eventually deteriorates.

W2: L1034 - 1036 (Now L1068 - 1075):

It means the probability source starts at uniform at t=0t=0, so that the probability that the sampled target equals to actual target rises from uniform 1/∣S∣1/\lvert\mathcal{S}\rvert to one as the posterior given current state proceeds over time. As it is generally unlikely to have zero uncertainty at inference time regarding the estimated target, this probability velocity typically stays non-zero and follows the given number of sampling steps.

AuthorsOpenReview ↗

Replies to Reviewer vwBo: Presentation and Experiments Weaknesses 3 - 8

Presentation

W3: L123 and L192 (Now L129 - 140 and L213 - 215):

The posterior at L123 conditions on the two terms following the notation in [2]. As we introduce the absorption indicator ff, for convenience we define sk=(a,f,C)s_{k}=(a,f,C) in L190 so that the denoiser also conditions on ff. Nevertheless, we add =Q1∣tkθ(a1j∣atk,ftk,Ctk)=\mathbb{Q}^{\theta}_{1\mid t_k}\bigl(a_1^j \mid a_{t_k}, f_{t_k}, \mathcal{C}_{t_k}\bigr) at L214 for further clarity.

W4: Fixed a1ja^{j}_{1}:

This is the notation in [2] (last 3 lines in page 7 for arxiv version): it samples an estimated target x1ix^{i}_{1} at each sampling step, and then update the current token based on this target. This estimated target is updated at every step (self-referential bias) which leads to inference instability [3]. Given the realised state sks_{k}, the denoiser defines a categorical distribution Q1∣tkθ(a^1j=z∣sk)\mathbb{Q}_{1\mid t_{k}}^{\theta}(\hat{a}_{1}^{j}=z \mid s_{k}) over possible token values z∈Sz\in\mathcal{S}. Under this distribution, we select the candidate token value to be absorbed by taking the argmax, instead of the state sks_{k}, and absorb it regardless of estimated target once the position reaches the entropy threshold: z^j=arg⁡max⁡z∈SQ1∣tkθ(a^1j=z∣sk)\hat{z}_{j}=\arg\max_{z\in\mathcal{S}}Q_{1\mid t_{k}}^{\theta}(\hat{a}_{1}^{j}=z\mid s_{k}), in order to minimise the probability of a wrong token estimated by the model. We understand that the notation might be a bit confusing, so we have updated the model estimation of target into a^1\hat{a}_{1} (or a^1j\hat{a}^{j}_{1} for each position) for further clarity (Now L166 - 167).

W5: L1286 (Now L1325 - 1328):

We conduct our comparison over ordering policy baselines with position mask, absorption schedule and other hyperpareters matched, except that the argmax is taken on different such metrics. This setup is to ensure that we isolate the policy difference for fair comparison. We also include Euler and corrected-flow as additional non-absorbing baselines for broad comparison. This paragraph has been updated accordingly.

W6: Frozen-rate jump probability:

We update it to the probability of jumping to another token given the same outflow rate λtj\lambda_{t}^{j}.

Experiments

W7: Hazard:

We update it to the probability of resampling a token from the denoiser at each step ckc_{k}. Figure 5 then shows the token changes are more likely as t goes to 1 with support that the estimated target at each step does not match current token.

W8: L1185 - 1186 (Now L1780 - 1784)

We arbitrarily perturb and make the indicator zero so the velocity is non-zero. The position resamples every step under a fair ablation with checkpoint controlled. Interestingly, perturbing the indicator in such pretrained model allows better performance than pretraining without any indicator, suggesting the absorption indicator carries some learned signals (App. F.1, Table 14; App. G.2, Table 25).

AuthorsOpenReview ↗

Replies to Reviewer vwBo: Questions 1 - 9

Questions

Q1:

Yes. For further clarity, we update that line to "dependence between coordinates absorbed together in the same step is not considered [3]" (Now L76 - 77).

Q2:

We update it with more straightforward terms for further clarity: "Absorbed positions are correct within the target data distribution, so that the current trajectory is still valid and able to reach the oracle state" (Now L79 - 81 and L249 - 253).

Q3:

We update schedule gradient to "velocity scaling factor", with a single line explanation in Sec. 2.2: "which is an inverse power of 1-t [2]".

Q4:

Number of positions to be absorbed at each step is pre-determined by some schedulers. Given this number to be absorbed at this step, we can select any unabsorbed positions, which would affect subsequent trajectory, hence motivating this work on ordering. We have updated the line to "The absorption count is based on the scheduler, which can be specified in advance. Given the number to be absorbed at this step bkb_{k}, we can select any unabsorbed positions, which could affect subsequent trajectory." (Now L201 - 203) to ease reading.

Q5:

We use cosine scheduler to align with standard masked diffusion work for comparison [1], with recent work showing its optimality [4]. We add a line to explain the motivation in L205 - 209, as well as additional results comparing against other schedulers in App. G.7, Table 32:

Count scheduleRefresh every step
Cosine0.832
Linear0.820
Sine0.816

Q6:

We update the pair to initial sequence a0a_{0} and target sequence a1a_{1} to ease reading (Now L880 - 881).

Q7:

We keep a_{0} to align with Eq. 14, but it can indeed be removed as the conditional distribution sums to one. We add another line following Eq. 16 with a_{0} term removed.

Q8:

We update that line (Now L1008 - 1011) to more straightforward usage of words: "The velocity is dependent on estimated target a^1j\hat{a}^{j}_{1} instead of actual target because actual target is unknown. This self-referential bias [3] leads to undesirable token changes even when the current state already matches the actual target."

Q9:

Jump probability is standard terminology in the CTMC field [5] including discrete flow matching. Our equation converts the transition rate into exponential waiting time to next transition to show that the probability to jump as t goes to 1 converges to 1. We add this line of intuition for clearer transition: "Holding the outgoing transition rate λtj\lambda_{t}^{j} fixed over a step of size h>0h>0 gives an exponentially distributed waiting time and the jump probability: 1−e−hλtj→11-e^{-h\lambda_t^j}\rightarrow1".

Reference

[1] Yang et al. Improving sampling for masked diffusion models via information gain. ICML, 2026.

[2] Wang et al. FUDOKI: Discrete flow-based unified understanding and generation via kinetic-optimal velocities. NeurIPS, 2025.

[3] Ouyang et al. Acceleration of diffusion language model through discrete average generator. Preprint, 2026.

[4] Zhang et al. The cosine schedule is Fisher-Rao-optimal for masked discrete diffusion models. Preprint, 2025.

[5] Caswell. Sensitivity Analysis of Continuous Markov Chains. Springer, 2019.

Reviewer_5o9hOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The authors address the topic of re-ranking/re-ordering in the domain of uniform discrete flow. The authors show (as prior work) that under certain condition intermediate steps sometimes provide the correct solution which is later revert causing a failure. They introduce generation order into uniform discrete flow through selective absorption, which fixes chosen predictions while preserving the uniform flow velocity at active positions. Their work is well evaluated on different benchmarks, well motivated, and well explained.

Critical strengths
  • Well defined motivation with clear limitations of prior arts and well defined approach
  • Well explained approach
  • Strong evaluation on different applications and domain
Critical weaknesses
  • Novelty: I think this is the main weakness of this paper, while not familiar with this specific research, the concept of re-ranking tokens or re-sampling tokens is quite common in the prior LLM research (as also cited by the authors), this could limit the novelty to a engineering contribution. The authors claim "LEDFLOW applies entropy-guided absorption to a revisable uniform flow with analysis of local prediction and dependence costs", could the author distinguish the contributions compared to the sota a little more? Is the main contribution specifically for uniform flows or is the claim more distinct and novel? What is the insight that your work provides over the prior art?
  • Can you explain the large difference in performance between datasets in Table 1?
  • The authors mainly use FUDOKI as the backbone, can you apply your method on other works? (Can you show some evaluation in this regard)
  • A core limitation the authors addressed is "Irreversibility", could the author extend this section by introducing specific application areas in which such decay happens, and if prior art has already aimed to address it?

Due to my unfamiliarity with the topic I am leaning to a weak-accept but would refer the comments of the other reviewers

Questions for authors

Have been asked under weakness section.

4 comments
AuthorsOpenReview ↗

Replies to Reviewer 5o9h: Weakness 1 - 3

Thank you for your comments. We have updated the paper with a new PDF file uploaded containing additional summarised related work and analysis (in blue) have our replies as follows.

1. Work novelty

This work focuses on uniform discrete flow, which differs from standard masked flow, as we sample on every position for each time step, and each position remains eligible for revision [1]. Ordering work exists in masked diffusion [2] as tokens are unmasked step by step, but uniform model resamples all positions at every time step, so that there is no natural ordering. Most ordering baselines choose which masked tokens to reveal, which we include a summarised related work for your reference (App. J, Table 39).

MethodSettingToken resampled?Focus
MaskGITMasked generationNoParallel decoding on confidence
Top-K marginMasked diffusionNoReveal order using Top-K margin
EB-SamplerMasked diffusionNoBlock dependence on entropy
Info-GainMasked diffusionNoReveal order on future uncertainty
ReMDMMasked diffusionYesRemasking to correct error
DiffusionGemmaUniform diffusionYesParallel refinement
FUDOKIUniform flowYesKinetic optimality velocity resampling
LEDFlowUniform flowYesSelective absorption with velocity resampling

The insight following the contribution is the theoretically and empirically backed selection decision between absorption in standard masked diffusion that prevents token correction and self-referencing biased [3] uniform diffusion which is prone to overcorrections.

2. Performance difference in Table 1

The score difference is due to different metrics for each benchmark in standard evaluation protocol [4, 5]: aggregate score for MME, LLM judge for MM-Vet and accuracy for remaining datasets. The performance difference attributes to the amount of text generated. In the following average length table, while the average ground truth word count is small, MMBench and MMMU do not explicitly ask for a single word answer. The model's explanation generated show to benefit the final answer more in the refining process of uniform flow. In contrast, POPE, MME and GQA explicitly ask for a single word answer, so the effect of ordering is not as explicit, as there is no ordering within single token. This is consistent with our claim where constrained reasoning is more benefitted by ordering (App. E.1, Table 6).

BenchmarkItemsMean ground truth lengthMean FUDOKI responseMean LEDFlow response
POPE5,1271.001.001.00
MME2,3741.001.001.00
MMBench4,3771.0014.2114.15
GQA12,5781.061.051.05
MMMU1,0501.0052.6452.65
MM-Vet21857.2753.0750.78

3. Transferability beyond FUDOKI

This work focuses on FUDOKI as it is the sota uniform flow model. Figure 4(b) showcase the transferability of entropy ordering to another pretrained backbone, i.e. LLaDA-8B, Dream-7B and MMaDA-8B, which retains the original masked sampler and checkpoints (attached as follows):

ModelGSM8KBBHTrackARCCSQA
LLaDA-8B+4.5+6.2+12+2.7−0.4
Dream-7B+12+18+4.8+7.0+5.0
MMaDA-8B+23+0.3+6.0+3.4+3.3

We further attach Table 11 information from Appendix E.4.2 which includes the details of Figure 4(b) for your reference (App. E.4.2, Table 11 in the revised manuscript).

ModelBenchmarkNet LEDFlow improvement over default sampling
LLaDA-8B-Instruct [22]GSM8K+0.045
Dream-7BGSM8K+0.121
MMaDA-8BGSM8K+0.228
MMaDA-8BBBH logical deduction+0.003
MMaDA-8BTracking shuffled objects+0.060
MMaDA-8BARC-Challenge+0.033
MMaDA-8BCommonsenseQA+0.033
LLaDA-8B-InstructBBH logical deduction+0.063
LLaDA-8B-InstructTracking shuffled objects+0.116
LLaDA-8B-InstructARC-Challenge+0.027
LLaDA-8B-InstructCommonsenseQA−0.004
Dream-7BBBH logical deduction+0.176
Dream-7BTracking shuffled objects+0.048
Dream-7BARC-Challenge+0.070
Dream-7BCommonsenseQA+0.050

As per your suggestion on more evaluation, we further include the following Dream-7B performance in GSM8K on different denoising steps, which shows that the benefit of entropy ordering also transfers to different denoising steps (App. G.8, Table 33).

Denoiser evaluationsDefault orderEntropy orderDifference
640.3650.436+0.071
1280.3710.492+0.121
AuthorsOpenReview ↗

Replies to Reviewer 5o9h: Weakness 4

4. Applications and prior works on irreversibility

Prior work has explored both reveal order and revision in masked diffusion (as attached in the related work table above), e.g. Top-K margin and Info-Gain select which masked positions to reveal while ReMDM allows revealed tokens to be masked again. Nevertheless, their application on uniform discrete flows where all tokens are repeatedly resample remains an open question, as shown in our implementations as absorbing state in Table 1 - 3 against pure uniform flow(consolidated below) (App. J, Table 39; main Tables 1 - 3):

Table 1

SamplerPOPEMMEMMBenchGQAMMMUMM-Vet
Uniform flow86.11485.473.957.634.338.0
Top K-margin83.51497.465.056.930.040.0
Info-Gain89.31503.065.156.429.137.5
LEDFlow (ours)89.01504.674.658.037.440.5

Table 2

SamplerGenEval SingleGenEval TwoGenEval CountGenEval ColorsGenEval PositionGenEval Attr.GenEval Overall
Uniform flow0.96250.83840.48750.88330.72000.63000.7536
Top K-margin0.94690.78540.41880.88560.62750.58000.7071
Info-Gain0.93120.77530.49060.87770.66000.55500.7150
LEDFlow (ours)0.95630.87120.52190.90160.71500.72250.7814

Table 3

SamplerMathVistaMathVerseGSM8KSudoku GeneratedSudoku NikoliSudoku ExtremeLatinGraphMolec.
Uniform flow0.2540.1100.0260.6100.4100.0780.6520.9360.828
Top K-margin0.2550.1200.0240.8200.7960.2710.9480.9510.887
Info-Gain0.2270.1190.0190.8040.7800.2310.9901.0000.914
LEDFlow (ours)0.2550.1230.0270.8650.8450.2690.9981.0000.906

Regarding the decay, in a Sudoku puzzle with a unique solution, fixing an incorrect digit makes the solution unreachable, despite Table 14 (key information as follows) shows that entropy ordering reduces fraction of incorrect absorptions from 25.4% under arbitrary ordering to 6.6% (now App. F.1, Table 14 in the revised manuscript).

ModelSolve rateIncorrect absorption rate
Uniform flow (no absorption)0.410—
Arbitrary-order absorption0.3400.254
Entropy order (LEDFlow)0.8450.066

Table 12 (key information attached as follows) further shows that for text-to-image generation, 12.5% of images under uniform flow satisfied the evaluation criterion at t=0.75 but eventually failed at t=1, compared with 2.5% under LEDFlow (now App. F.1, Table 12 in the revised manuscript).

MeasurementUniform Flow − LEDFlow
Token changes per position+1.81
Fraction of changes occurring after t = 0.75+0.34
OOD correct to wrong+0.100

Despite the wrong absorption rate is still non-negative, we see that LEDFlow provides net value to the performance as there are more correct tokens being resampled wrongly. Nevertheless, we will include in the limitations section regarding the wrongly absorbed tokens that can potentially bring net harms to the model.

References

[1] Gat et al. Discrete flow matching. NeurIPS, 2024.

[2] Yang et al. Improving sampling for masked diffusion models via information gain. ICML, 2026.

[3] Ouyang et al. Acceleration of diffusion language model through discrete average generator. Preprint, 2026.

[4] Wan et al. Corrected samplers for discrete flow models. ICML, 2026.

[5] Wang et al. FUDOKI: Discrete flow-based unified understanding and generation via kinetic-optimal velocities. NeurIPS, 2025.

[6] Chang et al. MaskGIT: Masked Generative Image Transformer. CVPR, 2022.

[7] Ben-Hamu et al. Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking. NeurIPS, 2025.

[8] Kim et al. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. ICML, 2025.

If you have any further concerns regarding our replies, please let us know. We are happy to discuss further in the remaining three days of rebuttal period. Thanks!

Thank you for the extensive response and sorry for the delayed response, I will keep my rating as is. The strengths of the paper only increase with the main comments. The main concerns of limited novelty remains but the novelty is reasonable in respect to the given research area, and finally up to the Area Chair to decide if this is sufficient.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.