acceptodds
Under review as a conference paper at ICLR 2027

FlexDLLM: Diffuse When Possible, Autoregress When Necessary

Abstract

Modern Large Language Model (LLMs) inference paradigms face a fundamental quality-efficiency trade-off: autoregressive (AR) decoding achieves strong generation quality but is inherently sequential, while diffusion-based LLMs (dLLMs) enable parallel decoding but often suffer from the degenerated performance. To overcome this dilemma, in this paper, we propose FlexDLLM, a flexible hybrid inference framework that adaptively switches between diffusion decoding and AR refinement. FlexDLLM is built on our key observation that inference errors in dLLMs typically stem from a few high-coupling tokens that are poorly suited for simultaneous updates. By applying AR refinement exclusively to these critical tokens, we can significantly boost performance while preserving the efficiency of highly parallel generation. Based on this insight, during inference, FlexDLLM dynamically alternates between two modes: a diffusion mode that efficiently decodes in parallel while autonomously identifying high-risk positions, and an AR mode that revises these difficult tokens conditioned on the global denoising state. Through this flexible switching mechanism, FlexDLLM strategically allocates sequential computation strictly to difficult, failure-prone spans. Experimental results demonstrate that FlexDLLM successfully marries the performance of AR with the speed of diffusion. It improves accuracy over the original LLaDA-8B-Instruct by 30.4% on GSM8K and 20.2% on HumanEval, while optimizing the accuracy-latency trade-off with a speedup on an H200 at comparable accuracy.

Primary area:
foundation or frontier models, including LLMs
Keywords:
model architecture design, inference acceleration

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_ih8kOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FlexDLLM alternates parallel diffusion with short autoregressive repairs. A trained diffusion adapter emits ASK tokens at predicted failure locations. A second adapter generates a repair conditioned on the denoising state. Teacher annotations and repair targets train the two stages. Experiments use LLaDA-8B-Instruct on six benchmarks.

Critical strengths
  1. Selective sequential repair is a concrete hybrid design. The shared backbone avoids keeping a second large inference model resident. Hardware measurements include adapter switching.

  2. The oracle analysis identifies a useful repair opportunity. Learned triggers outperform random and low-confidence triggers. Results span mathematical reasoning, code, and multiple-choice tasks.

Critical weaknesses
  1. The comparisons do not fully isolate selective repair from teacher supervision. Vanilla SFT helps, but its data and targets are not specified well enough to establish a matched distillation control. A model trained on the same repaired trajectories could explain part of the gain. All experiments use LLaDA 1.0. Six benchmarks broaden task coverage, but cannot establish transfer across model families or generations. Compatibility with newer block-diffusion and editing models remains untested. A matched comparison with CoDiLA [1] is also missing.

  2. The central analysis and reported configurations need clarification. Failure spans are labeled by o3 without a reported annotation audit. Table 3 uses the external teacher refiner. Its 85.1 score is not the deployed method’s 79.6 score. Repair limits also differ across descriptions: capacity 10 in Section 3.1 versus patch budget 30 in Table 5.

Questions for authors
  1. What happens when SFT uses the identical teacher-corrected trajectories and training budget? Please compare RG-Diff alone and the complete method. Do the repair gains persist on a newer backbone under the same supervision budget? If not tested, what limits the claimed applicability?

  2. Which refiner, patch capacity, and training configuration produced each main result? Please reconcile MBPP 51.2 in Table 1 with 54.2 in Section 4.1.

  3. How accurate are the failure-span labels under independent checking? How sensitive are the learned trigger and gains to annotation errors?

  4. The comparison with training-free correction methods is incomplete. WINO [2] revises suspicious drafts. ReMix [3] refines continuous token states. What advantage does learned AR repair provide under matched block sizes and generation limits? CoDiLA [1] is the closest hybrid comparator. The discussion should also distinguish joint prediction in CoDD [4] and dependency-aware selection in DEMASK [5]. Quality, total latency, and supervision costs should be separated.

[1] Locally Coherent Parallel Decoding in Diffusion Language Models

[2] Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs

[3] Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference

[4] Breaking the Factorization Barrier in Diffusion Language Models

[5] Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models

7 comments
AuthorsOpenReview ↗

Replying to Response to Reviewer ih8k [Part 1]

Response: We thank the reviewer for this insightful question and suggestion. To clarify the source of our performance gains, we conducted new experiments to explicitly disentangle the contribution of FlexDLLM's core mechanism from the supervision signal.

[Experiment: Disentangling FlexDLLM and Supervision.] Following the reviewer's constructive suggestion, we added two new baselines during the rebuttal:(1) Matched SFT: We train a baseline directly on the exact same teacher-corrected trajectories (from o3 and Qwen3) used by FlexDLLM, under an identical training budget (batch size 96, 200 steps).(2) RG-Diff alone: We run inference with the trained RG-Diff adapter but strictly block the <ASK> token (setting its logit to −∞). Consequently, the SC-AR repair is never triggered, reducing the decoding process to that of the original LLaDA.

MethodFine-tuning dataGSM8KMATHHumanEvalMBPPMMLU-ProARC-C
LLaDA-8B-InstructNone49.236.538.020.031.087.0
+ RG-Diff aloneStage I training data of FlexDLLM46.628.730.122.322.676.5
+ Matched SFTSame data as FlexDLLM52.844.847.134.338.288.6
+ FlexDLLMModel's own outputs, corrected by o3 and Qwen3-4B-Instruct79.653.458.251.254.589.3

From the result above, we can draw two conclusions: (1) RG-Diff alone degrades performance. When the model is restricted to only identifying error locations without the ability to correct them, its performance drops even below the original LLaDA baseline. This confirms that merely knowing where an error happens is useless without the capacity to fix it. (2) Matched SFT yields limited gains. Passively feeding the model correct trajectories is insufficient. Under this setting, the model never learns how to proactively discover and rectify its own errors during inference. These results show that every component of FlexDLLM is needed. FlexDLLM gets consistent gains because the model learns both to find errors and to fix them.

In addition, to reduce the reliance on external models, we also built an external-model-free version of FlexDLLM during the rebuttal. It uses simple rules to find and fix errors, and still brings large gains. Please see our response to Reviewer inWy (W1 (1) and W1 (2)) for details. We have added these experiments and analyses to Appendix F.1 and F.2 (Tables 7 and 8).

AuthorsOpenReview ↗

Replying to Response to Reviewer ih8k [Part 2]

W1 (2) & Q1 (2). Transfer to other backbones and newer block-diffusion/editing models, and comparison with CoDiLA.

Response: We thank the reviewer for this suggestion. During the rebuttal, we added experiments on two new backbones, together with a matched comparison with CoDiLA.

[Experiment on new model family: Dream-Coder-7B.] We follow the same pipeline as in the paper, and train FlexDLLM from scratch on Dream-Coder's own decoding trajectories. RG-Diff data: Dream-Coder generates answers for the first 10K Python problems in Magicoder-OSS-Instruct (4 tokens per step, length 256), and we record the decoding process. o3 then locates the first error and gives a fix. This gives 5,279 RG-Diff training samples. SC-AR data: We decode 2,000 problems with the trained RG-Diff adapter, and Qwen3-4B writes a patch for each <ASK>. This gives 5,994 patches to train the SC-AR adapter. The adapter sizes and training settings are the same as in the paper.

We test on HumanEval/HumanEval+ (generation length 768) and MBPP/MBPP+ (length 512), with 4 and 8 tokens per step. The baseline is the original Dream-Coder with the same decoder but no adapters. CoDiLA results are taken from its paper.

As shown below, FlexDLLM again performs best on Dream-Coder, and outperforms CoDiLA in 7 of the 8 settings under the same conditions. This is because FlexDLLM's find-and-fix framework is general and does not depend on a specific model architecture.

Tokens per stepMethodHumanEvalHumanEval+MBPPMBPP+
4Dream-Coder24.424.433.830.0
4CoDiLA51.848.855.645.5
4FlexDLLM58.255.362.654.3
8Dream-Coder8.58.514.012.5
8CoDiLA29.928.740.027.0
8FlexDLLM31.328.142.431.6

[Experiment on Newer block-diffusion model with self-editing: LLaDA2.1-mini.] To test compatibility with newer block-diffusion and editing models, we run experiments on LLaDA2.1-mini.

  • Data and training: The model generates answers for all 7,473 GSM8K training problems and the first 5,000 Python problems in Magicoder, using the official Quality mode, and we record the trajectories (including every self-edit). After o3 annotation, we get 398 (GSM8K) and 1,687 (Magicoder) RG-Diff training samples. RG-Diff is trained with a global batch size of 96 (4 epochs on Magicoder). Qwen3-4B then writes 1,095 and 4,996 patches, respectively, to train SC-AR with a global batch size of 128 for 3 epochs.
  • Decoder: LLaDA2.1-mini decodes differently from LLaDA 1.0: it generates from left to right in blocks of 32 tokens, and it edits tokens that were already written (self-editing). To work with this, we embed FlexDLLM's <ASK> trigger and repair into block-wise generation, with four rules: (1) <ASK> can only be emitted when filling a masked position, and self-editing cannot turn an existing token into <ASK>; (2) after the SC-AR patch is written back at the <ASK> position, all positions to its right are reset to masks and regenerated; (3) after the patch is written back, the original block structure is kept, and decoding resumes from the block where the patch ends; (4) the patch region is locked and cannot be changed by self-editing, so the repair is not overwritten.
  • LoRA placement: Since LLaDA2.1-mini uses an MoE architecture, we add LoRA only to the attention layers and leave the expert layers unchanged.

We test on GSM8K and HumanEval with generation length 256, and compare LLaDA2.1-mini with only its built-in editing against built-in editing + FlexDLLM. As shown below, FlexDLLM works well with this newest block-diffusion model with self-editing, and clearly improves both math and coding (+9.0 on GSM8K, +10.4 on HumanEval). This is because the find-and-fix skill learned by FlexDLLM fits well into different dLLM decoding frameworks, and it complements the model's built-in editing.

MethodGSM8KHumanEval
LLaDA2.1-mini (built-in editing only)70.348.8
LLaDA2.1-mini (built-in editing + FlexDLLM)79.359.2

We have added these results to Appendix F.5 (Tables 11 and 12). Thanks for the suggestion.

AuthorsOpenReview ↗

Replying to Response to Reviewer ih8k [Part 3]

W2 (1) & Q3. Quality of the o3 failure-span labels, and sensitivity to label errors.

Response: We thank the reviewer for pointing this out. During the rebuttal, we manually audited a random sample of o3's failure-span labels, and tested how sensitive FlexDLLM is to annotation errors.

[Manual audit of the labels.] We select 150 problems from the training set (50 math, 50 code, and 50 commonsense), and manually check whether o3's labeled failure span and failure step are correct. We find no clear labeling errors. For 19 problems (12.7%), the labeled failure step is one decoding step earlier or later than the true one, but the labeled span itself is still basically correct.

[Experiment: Sensitivity to annotation errors.] We add noise to the "where to emit <ASK>" labels in the Stage I training data, with two types of noise:

  • Position shift: we randomly move the <ASK> position of every sample left or right by 2, 4, 8, or 16 tokens. This simulates labels that are "roughly right but not precise".
  • Random replacement: we randomly pick 10%, 20%, or 40% of the samples and move their <ASK> to a completely random position. This simulates labels at "the wrong place".

For each setting, we retrain RG-Diff with exactly the same configuration as the original, and report the accuracy and the average number of <ASK> calls per question on GSM8K.

Label noiseNoneShift ±2Shift ±4Shift ±8Shift ±16Random 10%Random 20%Random 40%
GSM8K accuracy79.678.780.278.176.478.276.572.8

It can be seen that small position shifts (up to ±8 tokens) barely change the accuracy. When part of the labels are randomly wrong, the accuracy stays fairly stable at 10–20% noise and drops more at 40%. This shows that FlexDLLM is robust to small annotation errors. Therefore, the ±2-token shifts in only 12.7% of o3's labels have a negligible effect on the final performance: even shifting every label by ±2 tokens only changes the accuracy from 79.6 to 78.7.

We have added these experiments and analyses to Appendix F.4 (Table 10).

W2 (2). Table 3's 85.1 uses the external refiner, not the deployed method (79.6).

Response: We thank the reviewer for pointing this out. The 85.1 in Table 3 is the result when the model emits <ASK> by itself, but the patches are written by the external Qwen3-4B-Instruct. SC-AR is distilled from this model, so 85.1 can be seen as the upper bound of our method. Table 3 uses Qwen as the refiner for all rows because it compares how well different strategies choose the <ASK> positions. For a fair comparison, all strategies use the same model to write patches after <ASK>, so any difference comes only from where <ASK> is placed.

The deployed FlexDLLM uses only one model, and switches between two LoRAs for diffusion decoding and patch repair. Its GSM8K result is 79.6. This is our main result and how the method is used in practice. We have clarified this in the captions of Table 1 and Table 3, and in Appendix E (Table 6).

W2 (3). Inconsistent repair limits.

Response: We thank the reviewer for catching this inconsistency. We have checked the details: the patch length is 10 in both training and inference. The 30 in the appendix was a typo, and we have fixed it in the revised version (Table 5 and Appendix D.2.2).

Reviewer_2esfOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FlexDLLM, which switches between parallel diffusion decoding and local autoregressive refinement. A diffusion adapter learns to emit <ASK> at positions likely to cause errors. A second adapter then generates a short repair conditioned on the current diffusion state, inserts it into the sequence, and resumes diffusion decoding. Training uses annotated error locations and teacher-generated repairs. Experiments with LLaDA-8B-Instruct report substantial accuracy gains on six benchmarks and a better accuracy--latency trade-off than the original model.

Critical strengths
  1. I found the idea of spending sequential computation only on a few difficult spans intuitive and interesting. The analysis in Section 2 motivates this choice, and Table 3 suggests that learned intervention locations are more useful than simply selecting low-confidence tokens.

  2. The gains over the original LLaDA model are substantial across several tasks. Table 2 also measures wall-clock latency and memory, while the refinement-budget study shows that a relatively small amount of AR generation can recover much of the lost accuracy. These results are the main reason I lean toward acceptance.

Critical weaknesses
  1. The training comparison is not fully clear. FlexDLLM uses o3 error annotations and Qwen3-4B-Instruct repairs. The paper includes a Vanilla SFT baseline, but Appendix E does not establish whether it receives comparable teacher-generated supervision and training effort. This matters for separating the benefit of the decoding procedure from the benefit of distillation.

  2. The latency comparison is mainly against the original LLaDA model. Table 1 includes methods such as ReMeDi [1] and ReMDM [2], which also revise tokens during generation. Comparing them at the same unmasking rate does not imply equal runtime when their revision costs differ. Timing these baselines would make the claimed accuracy-latency advantage more convincing.

[1] Huang, Zemin, et al. "Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models." International Conference on Learning Representations. Vol. 2026. 2026.

[2] Wang, Guanghan, et al. "Remasking discrete diffusion models with inference-time scaling." Advances in Neural Information Processing Systems 38 (2026): 147282-147339.

Questions for authors
  1. What training data and budget does Vanilla SFT use? Does it receive the corrected outputs or other teacher supervision used by FlexDLLM, and is a comparison with matched supervision available?

  2. Are wall-clock results available for the stronger baselines in Table 1 under the same hardware and generation limits? Please also identify the task and decoding settings behind the abstract's 4.1x speedup.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 2esf [Part 1]

We would like to thank the reviewer for the careful review and the positive feedback. We are glad that the reviewer finds our idea of spending sequential computation only on a few difficult spans intuitive and interesting, and we address each concern below.

W1 & Q1. Training data and budget of Vanilla SFT, and a comparison under matched supervision.

Response: We thank the reviewer for this question and suggestion. Below, we clarify the Vanilla SFT baseline in the paper and add a new baseline with matched supervision.

[Vanilla SFT in the paper.] Vanilla SFT trains the model on the training sets of the same four datasets as FlexDLLM: GSM8K, MATH, Magicoder-OSS-Instruct, and CommonsenseQA. The only difference is that it uses the original reasoning chains in these training sets, not the outputs corrected by o3 and Qwen3-4B-Instruct. It is trained for 200 steps with batch size 96, the same training budget as FlexDLLM.

[New baseline: Matched SFT.] To compare under matched supervision, we add a Matched SFT baseline. It is trained directly on the same full trajectories corrected by o3 and Qwen3-4B-Instruct that FlexDLLM uses, with the same budget (batch size 96, 200 steps). As shown below, Matched SFT brings much smaller gains than FlexDLLM. This is becuase under matched SFT, model only sees correct trajectories, but never learns to find its own errors and fix them during inference. And FlexDLLM learns exactly this.

MethodFine-tuning dataGSM8KMATHHumanEvalMBPPMMLU-ProARC-C
LLaDA-8B-InstructNone49.236.538.020.031.087.0
+ Matched SFTSame data as FlexDLLM52.844.847.134.338.288.6
+ FlexDLLMModel's own outputs, corrected by o3 and Qwen3-4B-Instruct79.653.458.251.254.589.3

In addition, to reduce the reliance on external models, we also built an external-model-free version of FlexDLLM during the rebuttal. It uses simple rules to find and fix errors, and still brings large gains. Please see our response to Reviewer inWy (W1 (1) and W1 (2)) for details. We have clarified the Vanilla SFT setting in Appendix E, and added these experiments and analyses to Appendix F.1 and F.2 (Tables 7 and 8).

AuthorsOpenReview ↗

Response to Reviewer 2esf [Part 2]

W2 & Q2. Wall-clock latency of stronger baselines, and the setting behind the 4.1× speedup.

Response: We thank the reviewer for this suggestion. During the rebuttal, we timed more baselines to better show the accuracy–latency advantage of FlexDLLM.

[Experiment: accuracy and latency of more baselines.] For a fair comparison, we measure the accuracy and the average latency per question of each method on GSM8K and HumanEval, on a single H100 with batch size 1. For each run, we first warm up with 10 untimed samples, and then average over all remaining questions. All baselines decode 4 tokens per step (K=4).

As shown below, FlexDLLM stays on the accuracy–latency Pareto front among all baselines. Compared with ReMeDi and ReMDM, FlexDLLM (K=8) has both higher accuracy and lower latency. Compared with LLaDOU, which adds an extra decoding-order policy module, FlexDLLM (K=4) achieves higher accuracy with similar latency.

MethodHow it worksGSM8K AccGSM8K latency per sampleHumanEval AccHumanEval latency per sample
LLaDA-8B-Instruct (K=4)Original decoding49.21.66s38.02.69s
ReMeDiPredicts a confidence for each token, and re-masks filled tokens with low confidence62.12.54s34.24.55s
ReMDMRe-masks filled tokens with some probability during inference, then regenerates them52.22.00s32.63.98s
LLaDOUAdds a decoding-order policy module that scores each masked position to decide which ones to fill at each step77.12.12s53.14.29s
FlexDLLM (K=4)Parallel diffusion decoding, with AR repair at high-risk positions79.62.24s58.24.65s
FlexDLLM (K=8)Parallel diffusion decoding, with AR repair at high-risk positions78.41.23s49.42.79s

[Setting behind the 4.1× speedup.] The 4.1× speedup comes from the GSM8K subplot of Figure 4, i.e., GSM8K with generation length 256, measured on a single H200 GPU with batch size 1:

  • FlexDLLM decodes 8 tokens per step (K=8): 78.4% accuracy, 1.19 s per sample.
  • LLaDA-8B-Instruct decodes 1 token per step (K=1): 76.2% accuracy, 4.85 s per sample.

So the speedup is 4.85 / 1.19 ≈ 4.1×, with slightly higher accuracy. We have added these results to Appendix F.6 (Table 13), and made the 4.1× setting clear in the revised abstract. Thank you again for the helpful comments.

We hope our responses address your concerns, and we are happy to answer any further questions.

Thanks for your response and additional results. I think the matched-supervision SFD comparison and wall-clock measurement address my two concerns. The results on additional backbones also strengthen the evidence. I'll raise my score to clear accept.

Dear Reviewer 2esf,

Thank you very much for your positive response and for raising your score. We are delighted that our additional results and clarifications have addressed your main concerns. Your constructive feedback has helped us make our paper more solid and convincing.

We sincerely appreciate the time and effort you have devoted to reviewing our work.

Authors

Comments on the paper

8 comments
AuthorsOpenReview ↗

Response to Reviewer inWy [Part 1]

We would like to thank the reviewer for the time and the valuable feedback. We are glad that the reviewer finds our motivation strong and our dual-mode design efficient, and we address each concern below.

W1 (1). Heavy reliance on external models (o3 and Qwen3-4B-Instruct).

Response: We thank the reviewer for this concern. While our original implementation uses external models (o3 and Qwen3-4B-Instruct) to efficiently generate training data, FlexDLLM's core framework is agnostic to the source of the supervision signal and does not strictly require external models.

[Experiment: FlexDLLM without external models.] To demonstrate this, we conducted a new experiment replacing both external models with purely automated rules on GSM8K and MATH. Specifically, we localize the first reasoning error by deterministically matching numbers in the model's reasoning chain against the reference solution (replacing o3), and subsequently generate the correction by directly substituting this erroneous part with the corresponding correct step from the reference (replacing Qwen3-4B-Instruct).

As shown below, the rule-based version performs close to the original version without any external model (76.8 vs. 79.6 on GSM8K, 50.2 vs. 53.4 on MATH). This is because FlexDLLM only needs to know where the model's own reasoning first goes wrong and what the correct version is, which does not require a strong teacher model. So if the output can be checked by rules, errors can also be found and fixed automatically; for example, for coding tasks, we can run the generated code to locate the failing part and fix it with the reference solution. In these cases, FlexDLLM does not need any external model; o3 and Qwen3-4B-Instruct are only a general option for data without references or checkers.

MethodFine-tuning dataGSM8KMATH
LLaDA-8B-InstructNone49.236.5
+ FlexDLLM (original)Data generated with o3 and Qwen3-4B-Instruct79.653.4
+ FlexDLLM (rule-based)Data generated without external models76.850.2

We have added these results and the discussion to Appendix F.2 (Table 8), and thanks for your suggestion.

AuthorsOpenReview ↗

Response to Reviewer inWy [Part 2]

W1 (2). Does FlexDLLM really learn to self-correct, or just memorize the teachers' correction patterns?

Response: We thank the reviewer for raising this critical question. Based on our empirical results and new rebuttal experiments, we demonstrate that FlexDLLM acquires a genuine, generalizable self-correction capability rather than merely memorizing teacher distributions. We substantiate this from three perspectives.

(1) Efficacy across structurally diverse domains. A single trained FlexDLLM model concurrently improves performance on math, coding, and commonsense reasoning (Table 1 in the main text). These domains require fundamentally distinct correction strategies. For instance, rectifying intermediate arithmetic miscalculations differs drastically from fixing code syntax. The consistent improvement across such varied tasks indicates that FlexDLLM cannot be simply memorizing a singular, static correction template.

(2) Zero-shot generalization to unseen task formats. To rigorously rule out memorization, we evaluated FlexDLLM on two tasks whose required skills and correction patterns were entirely absent from the training set: Adversarial NLI (evaluating logical entailment) and IFEval (evaluating strict instruction following). As shown below, FlexDLLM yields consistent improvements on both tasks. This confirms that the model’s ability to identify and rectify errors transfers zero-shot to out-of-distribution scenarios, which would be impossible if it were merely mimicking seen training patterns.

MethodFine-tuning dataAdversarial NLIIFEval Prompt-strictIFEval Prompt-looseIFEval Inst-strictIFEval Inst-loose
LLaDA-8B-InstructNone49.442.543.153.553.8
+ FlexDLLMMath, coding, and commonsense data58.245.848.355.657.9

(3) Standard SFT on the same data is insufficient. If FlexDLLM only memorized the teachers' outputs, standard SFT on the exact same data should achieve similar results. To test this, we trained a Matched SFT baseline using the exact same corrected trajectories (by o3 and Qwen3) and training budget. As shown below, this SFT baseline performs much worse. This proves that simply showing the model correct reasoning paths is not enough; FlexDLLM succeeds because it explicitly learns how to find and fix its own errors during inference.

MethodFine-tuning dataGSM8KMATHHumanEvalMBPPMMLU-ProARC-C
LLaDA-8B-InstructNone49.236.538.020.031.087.0
+ Matched SFTSame data as FlexDLLM52.844.847.134.338.288.6
+ FlexDLLMModel's own outputs, corrected by o3 and Qwen3-4B-Instruct79.653.458.251.254.589.3

In summary, FlexDLLM teaches the model to find the key high-coupling positions and to think more carefully at these positions, and this find-and-fix skill can scale to different tasks and inputs.

We have added these experiments and analyses to Appendix F.1 and F.3 (Tables 7 and 9). Thanks for the suggeustion.

AuthorsOpenReview ↗

Response to Reviewer inWy [Part 3]

W3 & Q1 & Q2. Robustness beyond 512 tokens, and how <ASK> frequency scales with length.

Response: We thank the reviewer for this question and suggestion. We use 256/512 tokens in the main results because they are the standard settings for these benchmarks. All baselines use the same lengths, so the comparison is fair. To answer the questions, we run two experiments: we first extend the generation length for the same questions, and then let the actual content grow with the length.

[Experiment 1: longer generation length for the same questions.] We first keep the task unchanged: each prompt still contains one GSM8K question, and we only increase the generation length from 256 to 512, 1024, and 2048 tokens, with all other settings unchanged. We compare LLaDA-8B-Instruct (K=4) with FlexDLLM under two settings: at most 3 <ASK> calls, and no limit. We report the accuracy and the average number of <ASK> calls per question.

Generation lengthLLaDA (K=4)FlexDLLM (max 3 <ASK>)FlexDLLM (no limit)
25649.279.6 (1.23 <ASK>)79.8 (1.94 <ASK>)
51245.078.4 (1.68 <ASK>)79.2 (2.02 <ASK>)
102444.575.2 (1.81 <ASK>)78.5 (2.11 <ASK>)
204842.074.2 (1.83 <ASK>)76.4 (2.27 <ASK>)

We draw two conclusions:

(1) FlexDLLM stays robust as the generation length grows. From 256 to 2048 tokens, FlexDLLM drops by 5.4 points with the cap of 3 and only 3.4 points without a limit, while LLaDA drops by 7.2 points. FlexDLLM stays more than 30 points ahead of LLaDA at every length.

(2) The number of <ASK> calls grows only slightly with the window size. Without a limit, it goes from 1.94 to 2.27 while the generation length grows by 8×. A longer window does not add more reasoning to a single question: the answer stays about the same length, and the extra positions are mostly filled with end-of-sequence tokens. So <ASK> is triggered by actual errors, not by the window size.

[Experiment 2: longer outputs with more reasoning content.] A longer window alone does not create more chances to make errors, so it cannot test whether the <ASK> budget is enough when errors become more frequent. We therefore let the content grow with the length: we put m GSM8K test questions into one prompt (m = 1, 2, 4, 8) and ask the model to solve them in order in one pass, with the generation length set to 256, 512, 1024, and 2048 tokens, respectively. All methods decode the whole generation region as one block. We report the per-question accuracy and the average number of <ASK> calls.

Generation lengthLLaDA (K=4)FlexDLLM (max 3 <ASK>)FlexDLLM (no limit)
25649.279.6 (1.23 <ASK>)79.8 (1.94 <ASK>)
51229.869.2 (1.64 <ASK>)72.2 (2.02 <ASK>)
10246.266.5 (2.81 <ASK>)69.5 (4.81 <ASK>)
20480.050.1 (2.83 <ASK>)65.4 (7.47 <ASK>)

We draw two conclusions:

(1) FlexDLLM degrades much more slowly than LLaDA as the output gets longer. LLaDA's errors keep accumulating in long outputs. When the whole region is decoded as one block, it also tends to fill in end-of-sequence tokens too early, leaving no room for the later questions. FlexDLLM fixes errors in time at high-risk positions, which stops errors from spreading.

(2) When the content grows, the number of <ASK> calls grows roughly linearly with the length. At length 2048, FlexDLLM without a limit triggers 7.5 <ASK> calls per sample on average, and its accuracy is 15.3 points higher than with the cap of 3. This is because longer outputs have more chances to go wrong, so a larger <ASK> budget helps. Based on this, we can set the <ASK> budget to grow linearly with the generation length, which keeps the accuracy high while staying efficient.

Together, the two experiments show that FlexDLLM stays robust in both settings, and that the <ASK> frequency follows the amount of actual content rather than the window size. We have added both experiments and the analysis to Appendix F.8. Thank you again for the suggestion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.