acceptodds
Under review as a conference paper at ICLR 2027

When Reasoning Traces Overrun the Canvas of Diffusion LLMs

Abstract

A diffusion language model (dLLM) decodes into a fixed-length canvas, and supervised fine-tuning (SFT) on teacher traces shapes how much it writes before answering. We show that the SFT corpus affects such a student largely through overrun: reaching the end of the canvas without an answer, which we measure directly on saved generations of LLaDA-8B students. Holding teacher, problems and correctness fixed, prompting the teacher for terse rather than detailed traces raises GSM8K accuracy by 15.8 points at a 512-token canvas, and most of the gap (14.3 points) is attributable to overrun. A statistic computed before training, the share of training responses longer than the canvas, orders student overrun across corpora. A second check exposes a defect in the standard d1 pipeline: its 4,096-token SFT cap truncates about 90% of s1K examples and leaves only about one in ten with its answer, which teaches the student not to terminate. On LLaDA-8B-Base, where s1K SFT is no better than no SFT, replacing it with a corpus that fits the canvas improves accuracy by 8.8 GSM8K points, attributable to overrun, and by 13.7 MATH500 points, about half attributable to truncation. On LLaDA-8B-Instruct, which rarely overruns in-domain, s1K remains better in-domain; but on Countdown the s1K-trained student overruns on most problems, before and after identical GSM8K RL, and this accounts for the budget-fitting pipeline’s advantage there.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language models, masked diffusion, LLaDA, reasoning, chain-of-thought distillation, supervised fine-tuning, data curation, generation length, decoding budget, post-training, reinforcement learning from verifiable rewards

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pPL4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates how reasoning traces used for supervised fine-tuning interact with the fixed generation budget, or “canvas,” of diffusion language models. It distinguishes two potential sources of overrun: learning to produce reasoning that exceeds the available space, and learning not to terminate because training examples were truncated before their answers and ending tokens. Using LLaDA-8B-Base and LLaDA-8B-Instruct, the authors compare teacher-generated corpora and the s1K-based d1 pipeline. Their strongest controlled comparison uses the same 14B teacher and the same 5,661 problems: terse traces yield a 15.8-percentage-point GSM8K improvement over verbose traces at a 512-token budget, with 14.3 points assigned to overrun by their descriptive decomposition. They also find that the 4,096-token training cap removes the closed answer block from approximately 91% of s1K-1.1 examples. Replacing this corpus improves the Base student, while results on the Instruct student depend strongly on the task: s1K remains better on GSM8K, but the alternative corpus transfers better to Countdown, including after GSM8K reinforcement learning.

Critical strengths

The paper addresses a practically important evaluation problem: accuracy alone can conceal whether a model produces an incorrect answer or fails to deliver an answer within its budget. Its inspection of saved generations, separation of failure categories, and audit of training-time answer loss make the findings concrete and actionable. The same-teacher experiment is substantially stronger than simply comparing teachers of different sizes, and the persistence of its accuracy gap across checkpoints increases confidence that the observed effect is substantial. The paper also deserves credit for reporting unfavorable results: the proposed corpus does not consistently improve the instruction-tuned student, and mathematical fine-tuning harms broader capabilities. The statistical reporting is unusually transparent, with individual seeds, common-support comparisons, checkpoint sensitivity, and explicit limitations. The novelty is principally diagnostic and analytical. IGPO already proposed concise, length-aligned SFT traces, so this paper’s distinctive contribution is measuring the associated completion failures, quantifying answer-stripping preprocessing, and investigating when these problems influence downstream performance.

Critical weaknesses

My main concern is that the evidence establishes the observed failure pattern more convincingly than its causal explanation. The per-problem decomposition in Section 3.2 is valid accounting, but assigning a discordant example to overrun does not establish how much accuracy would be recovered by eliminating overrun. Likewise, similar accuracy on problems both students answer cannot exclude reasoning-quality differences on problems where one student overruns. The authors acknowledge this in Appendix C.1, but Section 5.3 still argues too strongly that an advantage in reasoning quality would necessarily appear as fewer wrong answers. The cleanest intervention also changes prompting, reasoning style, and intermediate content together with length; it has only one training run per arm and is evaluated at one budget. It therefore demonstrates that terse versus verbose supervision changes behavior, while isolating length itself remains incomplete. The s1K comparison has additional confounds: domain coverage, example count, system prompt, training schedule, and answer preservation all differ. Finally, the recommended corpus screen is insufficiently validated. Its 16-of-19 ordering statistic reuses a small collection of corpora, mean length produces identical rankings, and the suggested answer-retention threshold is not empirically established. Indeed, the screen would reject s1K even though it produces the better seeded Instruct GSM8K results.

Questions for authors

MATH500 uses a different budget-failure criterion from GSM8K because its parser can extract answers from unfinished outputs; this distinction should be visible wherever the attribution results are compared. The procedure for deciding that a generation is dominated by repetition, and for detecting a correct value inside an overrun, should be specified sufficiently for reproduction. Appendix D also shows that the strict correctness filter disproportionately discards correct 14B and 32B outputs containing monetary or thousands-separator formatting; the resulting topic-selection bias should be highlighted alongside cross-teacher comparisons. Checkpoint selection initially uses test performance, and the later robustness checks mitigate rather than fully replace the need for independent validation across seeds.

3 comments

Thank you.

1. The attribution is accounting, not causation. Agreed. To bound what overrun hides, we added answer forcing (Section 3.2, Appendix C.2, Table 7).

  • Base student, budget-fit vs s1K, three seeds. On the 411 problems the budget-fit student solves and s1K overruns, s1K is right on 34.8% when forced (CI 30.4–39.4). With both students forced, the gap falls from 8.8 to 4.8 points (CI 3.2–6.4).
  • Same-teacher intervention. The gap falls from 15.8 to 4.8 (CI 2.2–7.5). On the 204 problems the terse student solves and the verbose student overruns, the verbose student is right on 63.2% when forced.

We also corrected Section 5.3: a reasoning advantage now "should show at least partly as fewer wrong answers among answered problems"; the section states that both measures condition on answering, and that forcing cannot tell whether the remaining 4.8 points reflect weaker reasoning or loops that used up the canvas.

2. The isolation experiment: style, one run per arm, one budget.

  • Style: not isolated, and Section 5.1 and Appendix R say so. At L = 512 the evidence argues against a style-specific loop: the verbose student's overruns are long, non-repetitive derivations (median repeated-8-gram share 0.04), 52% already contain the correct value, and it is right on 63.2% when forced (point 1). At L = 1024 this no longer holds (next point).
  • One run per arm: still true for each student. The same two corpora give the same result on two more students: +69.8 points on Dream-7B (69.6 through overrun) and +66.2 on the AR student at L = 512 (all overrun).
  • One budget: now two (Appendix B, Table 5). On the first 512 test problems the gap is 21.3 points at L = 1024 (CI 16.2–26.2) against 12.9 at L = 512. Overrun accounts for 20.3 (CI 16.6–24.2), stopping for 0.8 and wrong answers for 0.2 (CI −2.9 to 3.1). We had predicted a smaller effect. On the larger canvas the verbose student overruns more (33.4% vs 25.0%), and 46% of its overruns are dominated by repetition (10% at L = 512 on the same problems). Part of the overrun at that budget is a loop, which this design cannot separate from length.

3. The s1K confounds. We ran the control (Section 5.3, Appendix L.1).

  • Repaired s1K: the same examples, prompt and schedule, with every training sequence keeping its answer and end of turn. It recovers 84% of the GSM8K gap over three seeds (+7.1, CI 5.2–9.1; overrun +6.5).
  • 32B-1k: a budget-fit subset matched to s1K in size, prompt and schedule. It scores level with the full budget-fit corpus (73.8 vs 73.3, seed 42).

With size, prompt and schedule ruled out on seed 42, domain and trace length remain. The repaired s1K is 1.4 points behind: overrun costs it 3.1, offset by wrong answers (1.5) and stopping (0.2); its traces still exceed the canvas.

4. Validating the corpus screen. We agree with all four points:

  • Reused corpora. The ordering now has a second student. On Dream with the same corpora, excluding s1K, the higher-ρ\rho corpus gives the student that overruns more in 9 of 10 pairs (Kendall τ = 0.80, p = 0.08). At L = 1024, all six pairs are concordant (Appendix K). These are still the same corpora.
  • Mean length. Agreed. Section 3.4 already calls ρ\rho a length screen and says mean length orders our corpora exactly as ρ\rho does; we claim nothing beyond length.
  • Threshold. We set none: our corpora sit at 9% and 100% ϕ~\tilde\phi (Section 3.4).
  • The screen rejects s1K. Section 3.4 now states that the screen predicts overrun, not accuracy. It rejects s1K, which is the better GSM8K corpus for LLaDA-8B-Instruct, because that student does not overrun in-domain. It is also the worse corpus for Dream-7B-Instruct, which does overrun (−20.4 points).

Questions.

  • MATH500 criterion. Section 3.2 states it, and every MATH500 attribution carries an asterisk.
  • Repetition and correct value. Appendix P now defines both (repeated word-8-gram share above 0.5; the reference answer as a standalone number after removing thousands separators). The scoring code is in the release.
  • Filter topic bias. Section 4 now reports it next to the corpora: monetary problems are 24.6% and 26.0% of the 14B and 32B corpora, against 29.6% of GSM8K train. Appendix D shows that the skew does not favour the budget-fit corpus on test: its lead over s1K is +13.8 on monetary problems and +6.6 on the rest.
  • Checkpoint selection. Agreed that the LLaDA protocol picks steps on test data. Every run added in revision chooses each run's step on 256 fixed GSM8K problems and is scored on the other 1,063: Dream, repaired s1K, 32B-1k and AR. For the LLaDA base s1K comparison, Appendix F also selects each seed's step on held-out data. We did not redo the LLaDA protocol for the other comparisons.

We agree with your reading of the contribution: it is diagnostic and analytical. The introduction and Related Work state its relation to IGPO.

Reviewer_pPL4OpenReview ↗

Comments on author response

Thank you for the detailed response and additional experiments. The three-seed repaired-s1K control and answer-forcing results substantially address my main concerns about confounding and the interpretation of overrun. The Dream and AR experiments also strengthen the scope of the findings, while the revised discussion appropriately acknowledges the remaining entanglement between length, style, and nontermination. I encourage the authors to maintain this distinction throughout the paper: answer forcing recovers approximately 45% of the Base comparison gap and 70% of the same-teacher gap, supporting a substantial recoverable component without establishing that the entire descriptive attribution is causal. Please clarify the denominator behind the reported 84% recovery from repairing s1K, and present the relevant absolute accuracies under matched evaluation and checkpoint-selection protocols.

Thank you. Each point is now addressed in the revised PDF.

1. The denominator of the 84%. It is the budget-fit corpus's paired gain over s1K-1.1 on the same 1,063 GSM8K problems (those not used for step selection), at L = 1024, over the same three seed pairs. The repaired s1K gains 7.09 points and the budget-fit corpus 8.47 (CI 6.6–10.3), so the share is 84% (CI 65.5–104.5%, resampling problems jointly for the three arms). In that comparison only the repaired runs had their steps chosen on the 256-problem screen; s1K-1.1 and the budget-fit corpus were scored at their reference steps.

2. Matched checkpoint selection. Every run's step is now chosen by one rule: highest accuracy on the fixed 256-problem GSM8K screen at L = 512, earliest on ties, computed from the saved full-test generations. One evaluation was missing (budget-fit, seed 2, at its newly selected step), and we ran it.

Selected steps:

CorpusSeed 42Seed 1Seed 2
s1K-1.1500500500
Repaired s1K1,5005002,460
Budget-fit3,5002,0001,500

For s1K-1.1 the rule picks the reference step in every seed; for the budget-fit corpus it changes seeds 1 and 2.

GSM8K accuracy at L = 1024 on the 1,063 problems (mean ± s.d. over three seeds; the repaired s1K is screen-selected in both columns):

CorpusMatched selectionReference steps
s1K-1.164.4 ± 4.464.4 ± 4.4
Repaired s1K71.5 ± 2.971.5 ± 2.9
Budget-fit72.0 ± 1.272.9 ± 2.4

Under matched selection, paired on the same problems and seeds:

  • Repaired s1K vs s1K-1.1: +7.1 points (CI 5.2–9.1), unchanged because the s1K-1.1 steps do not change: overrun +6.5, stopping +0.3, wrong answers +0.3.
  • Budget-fit vs s1K-1.1: +7.6 points (CI 5.8–9.3).
  • Recovered share: 94% (7.09 of 7.56, CI 74.0–118.0%), against 84% (7.09 of 8.47, CI 65.5–104.5%) at the reference steps.
  • Repaired s1K vs budget-fit: 0.5 points behind (CI −1.2 to 2.1): overrun costs it 2.4 (CI 1.6–3.2) and stopping 0.1, while wrong answers favour it by 2.0 (CI 0.6–3.4).

Under either protocol, keeping s1K's answers recovers most of the gap, and neither interval excludes full recovery. What remains in the point estimate is overrun from traces that still exceed the canvas (ρ = 99.9%). The repair still covers only the base student on GSM8K (Appendix R).

3. Recoverable versus descriptive. We agree and keep the two apart. Section 3.2 now states that answer forcing closes about 45% of the base-student gap (8.8 to 4.8 points) and 70% of the same-teacher gap (15.8 to 4.8), a substantial recoverable component, while the attribution itself stays descriptive. The limitations say the same and add that forcing recovers nothing on Countdown.

Changes: Section 3.2; Section 5.3 (matched share next to the 84%); Appendix L.1 and Table 26 (both protocols, each run's step in parentheses); limitations (Section 6).

Reviewer_KQ5bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how SFT reasoning-trace length and truncation affect diffusion LLMs with fixed output budgets. Experiments on LLaDA-8B link accuracy losses to “overrun”—exhausting the output canvas without an extractable answer—and identify answer loss caused by truncation in the studied d1 pipeline. The authors propose two corpus-level diagnostics and examine when budget-aligned training data reduces overrun and improves performance, with benefits varying across tasks and student variants.

Critical strengths
  1. Practically useful diagnostic insight. The paper identifies a concrete preprocessing issue: the studied SFT truncation removes answers and termination signals from most s1K examples. Its inexpensive corpus-level checks offer useful guidance for inspecting training data against both training and inference limits.

  2. Detailed analysis beyond aggregate accuracy. The paper separates overrun, stopping without an answer, and incorrect answers, with per-seed reporting and comparisons on commonly answered problems. It also reports cases where budget-aligned data does not improve performance, providing a nuanced account rather than claiming that shorter traces are universally better.

Critical weaknesses
  1. Limited cross-model validation. All student experiments use LLaDA-8B-Base and LLaDA-8B-Instruct, leaving the generality of the findings across diffusion model families unclear. Replicating the core interventions on another model, such as Dream-7B, would strengthen the conclusions. An autoregressive control under the same output cap would also help distinguish architecture-specific behavior from general budget and truncation effects.

  2. Missing direct validation of the truncation mechanism. The comparison between s1K and the budget-fit corpus also changes domain, dataset size, system prompt, and training budget. A more direct test would retain the same s1K examples while modifying truncation to preserve answers and EOS tokens, with other training settings matched. Without this control, the claim that answer-stripping truncation causes non-termination remains insufficiently isolated.

Questions for authors

see the weaknesses section

1 comment

Thank you. Both requested experiments are now in the paper. Both support the account, with differences we report.

W1: validation across model families.

Dream-7B (Section 5.7, Table 3, Appendix K). Same corpora, d1 SFT and evaluation code, hyperparameters and sampler. Each run's checkpoint is chosen on 256 GSM8K problems that are not scored. Every base-student effect has the LLaDA sign and is larger:

  • Length intervention (L = 512): terse 76.7 vs verbose 6.9 on GSM8K. Overrun accounts for 69.6 of the 69.8 points (CI 66.9–72.3).
  • Corpus ordering: at L = 1024, student overrun rises with the over-budget rate ρ\rho across all four corpora.
  • Budget-fit vs s1K, three seeds: 78.1 ± 1.3 vs 13.8 ± 9.4 on GSM8K and 37.9 ± 5.2 vs 2.1 ± 1.3 on MATH500 (Welch p = 0.006 and 0.005). These two survive a Holm correction over all nine tests.
  • Instruct: after s1K SFT, Dream-Instruct overruns on 32.0% of GSM8K problems, and the budget-fit corpus wins in-domain by 20.4 points: overrun +23.1 (CI 20.6–25.8), stopping +0.9, wrong answers −3.7.

One difference: the form of s1K's overrun depends on the student. On Dream, the s1K students' overruns are long, mostly non-repetitive reasoning, and 68–72% of them already contain the correct value. Forced to answer, the Dream s1K students are right on 75.7% of the problems they lose to overrun, against 34.8% on LLaDA. So the low s1K scores on Dream reflect overrun, not a broken port: the same code trains the budget-fit student above un-finetuned Dream (78.1 vs 66.8 on GSM8K), and Appendix K checks the one alignment change the port makes (masked cross-entropy 1.34 nats with the logit shift, 12.05 without).

Autoregressive control (Section 5.7, Appendix L.2, Table 27). Qwen2.5-7B, the model Dream is initialized from, trained with the same LoRA settings, corpora, 4,096-token truncation and step budgets, decoding at most L new tokens:

  • GSM8K, L = 1024: budget-fit beats s1K by 12.6 points (CI 10.0–15.2): overrun +15.5, wrong answers −2.9.
  • MATH500, L = 1024: budget-fit wins by 13.6 points (budget failures +20.8, wrong answers −7.2).
  • L = 512: terse beats verbose by 66.2 GSM8K points and 29.2 MATH500 points, all of it through overrun.

So overrun is not specific to diffusion. Its repetitive form is specific to LLaDA: only 7% of the AR s1K student's overruns are dominated by repetition. Unlike either dLLM, the AR student gains from s1K SFT over no SFT (+8.2 GSM8K points). On MATH500, when both students finish, the s1K-trained one is the more accurate.

Limits (Appendix R):

  • one seed per AR corpus;
  • the Dream Instruct comparison is a single run on GSM8K only.

W2: direct validation of the truncation mechanism. We ran the control you describe (Section 5.3, Appendix L.1, Table 26).

The repaired s1K keeps everything from the original s1K runs:

  • the same 1,000 s1K-1.1 examples;
  • d1's system prompt and chat template;
  • the 2,460-step schedule;
  • three seeds.

Only the truncation changes. For the 915 examples within 32 tokens of the 4,096-token cap (913 exceed it), the middle of the reasoning is removed, so every training sequence keeps its answer block, end of turn and end-of-sequence padding. The share of examples that keep their answer after the cap (ϕ~\tilde\phi) goes from 9.0% to 100%, and ρ\rho at L = 1024 stays at 99.9%. Each run's step is chosen on 256 fixed GSM8K problems and scored on the other 1,063.

  • Against the original s1K: GSM8K accuracy goes from 64.4 to 71.5 (+7.1, CI 5.2–9.1). Overrun accounts for +6.5 of it, stopping for +0.3 and wrong answers for +0.3. That is 84% of the 8.5-point gap to the budget-fit corpus.
  • Against the budget-fit corpus: the repaired s1K is still 1.4 points behind (CI −0.3 to 3.1): overrun costs it 3.1 (CI 2.3–3.9), while wrong answers favour it by 1.5 and stopping by 0.2. The repaired traces are still longer than the canvas, which is the other failure form ρ\rho flags.
  • Size, prompt and schedule (32B-1k, seed 42): 1,000 random examples of the budget-fit corpus, trained on s1K's prompt and 2,460-step schedule, score 73.8 against 73.3 for the full corpus (paired +0.5, CI −2.4 to 3.4). Size, prompt and schedule explain none of the seed-42 gap.

Limits (Appendix R):

  • the repair covers the base student on GSM8K only;
  • 32B-1k is a single seed.

Changes: the abstract, finding 3, Section 5.3, Appendix L.1.

If these two experiments address both weaknesses, we would be grateful if you would reconsider your rating; if something else stands in the way, please tell us before the discussion closes.

Reviewer_63sJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors examine the effects of applying SFT to MDMs with datasets that contain training examples with responses larger than the MDM’s canvas. They trace generations and classify them systematically into four outcomes: correct, wrong, overrun, and stopped. They introduce two measures: first, (\rho) measures the percentage of examples in the training datasets where the responses exceed the inference budget. \tilde{\phi} measures the percentage of answer blocks that fall within the training-token truncation budget. They show that for a base model (llada-8b-base), fine-tuning with such a dataset causes the model to overrun the inference canvas without an answer. More such examples lead to more frequent overruns. They also show that an instruct model is somewhat immune to this phenomenon. Interestingly, while the fine-tuned instruct model doesn’t exhibit the phenomenon in GSM8K, the behavior survives in the Countdown dataset. They show they can improve the performance by introducing a budget-fit training corpus on the base model.

Critical strengths
  1. They identify a defect in a frequently used pipeline(d1). S1K has small \phi and large \rho, and fine-tuning with it can instill undesirable behavior that hurts performance. In Countdown, even the Instruct model shows more overruns at inference budget L=1024.
  2. They also show that trace length alone can change accuracy. They use a DeepSeek-R1-Distill-Qwen-14B teacher to generate traces “concisely” and “explain your reasoning”. They ensure the traces are correct and fit the training sequence limit. On a shorter canvas (L=512), the model trained with longer traces loses by 15 pp, and almost all of it is because it doesn't have enough canvas to produce an answer.
Critical weaknesses
  1. “In-domain” is not properly defined in the paper.
  2. An instruction-tuned model performs better on GSM8K after fine-tuning with S1K. In Countdown, the fine-tuned model performs worse. Predicting which datasets will have overrun issues, even when dormant in others, will be useful and interesting. 5.4 attributes this to Countdown inviting long search; however, more systematic evaluation is necessary to establish this.
  3. The paper's organization is hard to parse.
Questions for authors

N/A

1 comment

Thank you. Each of the three weaknesses now has a change, below.

W1: "in-domain" is not defined. Agreed. Section 4 (Evaluation) now defines it:

  • GSM8K is in-domain, because the teacher corpora and the RL stage use its training split.
  • MATH500 is harder mathematics outside that split.
  • Countdown is out-of-domain: an arithmetic search task on which no student is trained.

The paper uses the term only in this sense.

W2: predicting where the dormant habit surfaces; "Countdown invites long search" needs systematic evaluation. Agreed: the submitted claim rested on one task contrast. New Appendix J tests a predictor that needs no training: the un-finetuned LLaDA-8B-Instruct's own answer length on the same problem. The outcome is the share of problems on which the s1K student's response is incorrect and its turn never closes within the canvas (L = 1024).

TaskUn-finetuned median answer length (tokens)s1K failures
GSM8K2692.0%
HumanEval (not executed; any unclosed turn)2695.5%
MATH50046116.5%
Countdown1,02248.0%
Sudoku (new run)1,02243.4%
  • Across tasks. The un-finetuned model's median answer length and the s1K student's failure rate rise together (table above). On Sudoku the un-finetuned and budget-fit students fail on 0%.
  • Across difficulty. Within MATH500, failures rise from 8.5% at level 1 to 27.1% at level 5.
  • Across problems. The signal is modest where length has a range (AUC 0.63 on GSM8K and 0.67 on MATH500, both p < 1e-4) and unresolved on HumanEval (0.57, nine failures). There is none on Countdown (0.47) or Sudoku (0.52), where the un-finetuned model's answers nearly all fill the canvas.
  • Budget-fit student. It fails on at most 0.8% of problems on any task.

So the habit surfaces where the task demands long outputs, and one evaluation of the un-finetuned model predicts where: well across tasks, modestly within a task.

The student matters too. Dream-7B-Instruct does not keep the habit dormant: after s1K SFT it overruns on 32% of GSM8K problems and loses in-domain by 20.4 points (Section 5.7).

Changes: Section 5.4, finding 5 in the introduction, Appendix J (Table 19, with the Sudoku row).

W3: the organization is hard to parse.

  • The introduction now lists the findings as numbered items in the order of Section 5, each with its section pointer.
  • The contributions paragraph is shorter.
  • Definitions used later (in-domain, the MATH500 failure rule) are stated where they are first used.

If particular passages were hard to follow, please point us to them and we will rewrite them.

If these changes address W1–W3, we would be grateful if you would reconsider your rating. If a soundness concern beyond W2 remains, please tell us before the discussion closes and we will address it.

Comments on the paper

1 comment

Summary of the revision. Each item below names the sections that changed. The main text still fits in 10 pages. Section, table and appendix numbers refer to the revised PDF.

  1. Second dLLM family (KQ5b). Dream-7B Base and Instruct, with the same corpora, d1 code, hyperparameters and sampler (Section 5.7, Appendix K). Every base-student effect has the same sign as on LLaDA and is larger:
    • the same-teacher length intervention gives +69.8 GSM8K points at L = 512, 69.6 of them through overrun;
    • budget-fit vs s1K over three seeds: 78.1 ± 1.3 vs 13.8 ± 9.4 on GSM8K and 37.9 ± 5.2 vs 2.1 ± 1.3 on MATH500.
  2. Autoregressive control (KQ5b). Qwen2.5-7B, the model Dream is initialized from, trained with the same settings and decoding at most L new tokens (Section 5.7, Appendix L.2):
    • budget-fit beats s1K by 12.6 GSM8K points at L = 1024 (CI 10.0–15.2): overrun +15.5, wrong answers −2.9;
    • terse beats verbose by 66.2 points at L = 512, all of it through overrun.
  3. Repaired s1K (KQ5b, pPL4). The same 1,000 s1K examples, prompt and schedule, with the middle of the reasoning cut so that every training sequence keeps its answer and end of turn (Section 5.3, Appendix L.1):
    • over three seeds, GSM8K accuracy rises from 64.4 to 71.5 (+7.1, CI 5.2–9.1). That is 84% of the gap to the budget-fit corpus.
    • a budget-fit subset matched to s1K in size, prompt and schedule (32B-1k, one seed) scores level with the full budget-fit corpus.
  4. Answer forcing (pPL4). At the budget, every overrun is cut and an answer is forced (Section 3.2, Appendix C.2, Table 7). This gives a lower bound on how many correct answers the overruns hide.
  5. Length intervention at a second budget (pPL4). At L = 1024 on 512 test problems, terse beats verbose by 21.3 points (12.9 at L = 512 on the same problems), 20.3 of them through overrun (Section 5.1, Appendix B, Table 5).
  6. Where the dormant habit surfaces (63sJ). The un-finetuned model's own answer length predicts where the s1K habit appears. Tested across tasks, difficulty levels and problems, including a new Sudoku run (Section 5.4, Appendix J).
  7. Presentation (63sJ, pPL4).
    • "In-domain" is now defined (Section 4).
    • The findings are now a numbered list in the introduction, in the order of Section 5, each with a pointer to its section.
    • The MATH500 criterion (Section 3.2) and the corpus-screen caveats (Section 3.4) are stated where they are used.
  8. Release. Code, corpora as trained, and every scored generation (link in the abstract and Appendix Q): https://anonymous.4open.science/r/ConciseDistill/

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.