acceptodds
Under review as a conference paper at ICLR 2027

Beyond the Canvas: Efficient dLLMs via Self-Guided CoT Compression and Suffix Sparsification

Abstract

Diffusion large language models (dLLMs) have emerged as a promising alternative to autoregressive generation. Existing acceleration methods mainly improve per-step decoding parallelism, but largely overlook redundancy at the trajectory level. dLLMs iteratively unmask tokens on a fixed-length masked sequence (canvas), which is typically set relatively large to accommodate unpredictable Chain-of-Thought (CoT) generation. We observe that such a design can lead to two key problems that affect the efficiency of dLLM reasoning: (i) the redundancy in reasoning CoT induced by an oversized canvas; and (ii) the redundancy in suffix tokens as context. To alleviate these problems, we propose the Beyond-the-Canvas dLLM (BC-dLLM), a framework with two complementary strategies: Self-Guided CoT Compression (SGC) and Progress-Aware Suffix Sparsification (PASS). Specifically, the SGC strategy aims to decouple the reasoning length from the canvas size, enabling the model to generate compact CoTs on large canvases. SGC uses the dLLM itself to construct variable-length CoT supervision data and applies conditional supervised fine-tuning to internalize this compact reasoning behavior on large canvases. The design of the PASS strategy is based on our insight that information gain is typically concentrated in suffix tokens adjacent to the current decoding block and diminishes as reasoning progresses. Hence, PASS shrinks the suffix window as reasoning progresses to reduce per-step computation on large canvases. Experimental evaluations against multiple baseline models on four reasoning benchmarks show that our method significantly reduces decoding steps while maintaining comparable performance to the baselines. When applied to LLaDA-8B-Instruct, our method achieves a latency speedup on the GSM8K dataset while maintaining comparable performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language models, Chain of Thought Compression

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZjpSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies the inefficiency in dLLMs caused by oversizes generation canvases. To address this, it proposed BC-dLLM with two components. SGC truncates the model's own long reasoning trajectories and regenerates the remainder within a restricted canvas, then fine-tune the model. PASS estimates decoding progress from <EOS> probabilities and shrinks the visible suffix window accordingly. Experiments on LLaDA-8B-Instruct, LLaDA-1.5 and Dream-7B across four math reasoning benchmarks show that this method reduces decoding steps by about 13x and latency by about 10x.

Critical strengths
  1. A well-motivated perspective. Instead of targeting step-level efficiency, this paper points out trajectory-level inefficiency caused by the oversized canvas.
  2. Two simple components. SGC reduces the number of tokens to generate and PASS reduces the cost per step. PASS is training-free and SGC needs only about 4K training steps.
  3. Comprehensive experiments. This paper conducts the experiments on three models across four benchmarks. The result shows that it reduces decoding steps by roughly 13–15x and latency by roughly 10x.
Critical weaknesses
  1. The sources of the speedup are not disentangled. There are several separate factors contribute to the reduction: confidence-threshold parallel decoding, <EOS> early termination, SGC and PASS. It is unclear the concrete contribution for the proposed methods.
  2. Accuracy drops consistently, not comparable. The performance for all benchmarks is worse than the original model. This should be characterized as an accuracy-efficiency trade-off.
  3. The simple length-control baselines are missing. Natural and simpler alternatives to induce the verbose CoTs should be running the original model with Fast-dLLM decoding and <EOS> early stopping on a smaller canvas.
Questions for authors
  1. Could you report the performance for: (a) SGC only, (b) PASS only, (c) SGC+PASS applied to the original model?
  2. Could you give more results on out-of-domain tasks (e.g., GPQA or MBPP for code) when SGC is trained only on GSM8K and MATH-500?
  3. What is the width of the restricted canvas uses to regenerate the fast-thinking segment? How is the truncation point sampled?
4 comments

We thank the reviewer for the thoughtful comments and suggestions. Our responses are as follows.

W1 & Q1: The sources of the speedup are not disentangled. Please report the performance of (a) SGC only, (b) PASS only, and (c) SGC + PASS applied to the original model.

Thank you for this valuable suggestion. To disentangle the sources of speedup, we evaluate (a) SGC only, (b) PASS only (applying PASS directly to the original base model), and SGC + PASS on GSM8K across different confidence thresholds (accuracy (%) with decoding steps in parentheses):

Table 1. Ablation study disentangling SGC and PASS on GSM8K across confidence thresholds.

Method0.90.80.70.6
SGC only81.6 (44)79.1 (33)74.5 (26)70.3 (20)
PASS only82.1 (89)81.1 (61)80.3 (52)74.9 (38)
SGC + PASS81.0 (39)78.9 (30)74.5 (24)70.1 (20)

The results clearly illustrate the distinct and complementary roles of both components:

  • SGC only and PASS only:SGC onlyaccounts for the majority of the step reduction (e.g., reducing steps from 89 to 44 at a threshold of 0.9) by training the model to generate concise CoT trajectories with high parallelism. Meanwhile, PASS only applied to the original base model maintains high accuracy (82.1% at 0.9) while reducing per-step computation via dynamic <MASK> pruning, though it still requires more steps (89 at 0.9) because its CoT trajectories remain uncompressed.
  • SGC + PASS combines both benefits, further reducing decoding steps from 44 to 39 (at 0.9) and from 33 to 30 (at 0.8) alongside lower per-step FLOPs, while achieving nearly the same accuracy as SGC only (e.g., 74.5% for both at 0.7). We will include this ablation table in the revised manuscript.

W2: Accuracy drops consistently, not comparable. This should be characterized as an accuracy-efficiency trade-off.

Thank you for your constructive comment. We agree to refine our wording in the revision to explicitly acknowledge the accuracy–efficiency trade-off relative to the Base model. We would also like to place our contribution in the context of different acceleration baselines:

In our experiments, besides the Base model, we compare against training-free acceleration frameworks such as Fast-dLLM and KLASS, as well as the training-based method dParallel. While Fast-dLLM and KLASS exhibit smaller performance drops, their acceleration gains are relatively limited. Conversely, training-based approaches can achieve much greater speedups at the cost of a moderate accuracy trade-off. Notably, compared to dParallel, our method achieves lower inference latency while simultaneously delivering higher accuracy. This demonstrates that our approach offers substantial practical value by advancing the overall efficiency–performance boundary, rather than representing a trivial accuracy–efficiency trade-off.

W3: The simple length-control baselines are missing.

Thank you for suggesting this baseline comparison. Here, we evaluate a prompt-based length-control baseline (Fast-thinking prompt only) that incorporates a CoT-shortening instruction into the prompt to encourage more concise reasoning. The comparison on GSM8K across different confidence thresholds is presented below:

Table 2. GSM8K results with LLaDA under different confidence thresholds (accuracy (%) with decoding steps in parentheses).

Method0.90.80.70.6
Fast-thinking prompt only76.2 (56)75.1 (47)73.1 (39)61.8 (31)
Ours81.0 (39)78.9 (30)74.5 (24)70.1 (20)

As the results demonstrate, although simple prompt constraints can shorten the model's CoT output, they lead to severe performance degradation compared to our method. For example, at a confidence threshold of 0.9, the prompt-only baseline achieves only 76.2% accuracy (with 56 steps), whereas our method reaches 81.0% (+4.8%) with just 39 steps.

Q2: Provide more results on out-of-domain tasks.

Thank you for your constructive comment. Our primary goal is to improve long-CoT generation efficiency while preserving the parallel decoding capability of dLLMs, which is why our main experiments focus on GSM8K and MATH-500. Since most standard general-domain benchmarks evaluate answer-option log-likelihoods rather than free-form generation efficiency, we additionally evaluate two free-form out-of-domain tasks: TruthfulQA-Gen and CNN/DailyMail.

Table 3. Out-of-domain evaluation on free-form generation tasks (score with speedup in parentheses).

MethodTruthfulQA-GenCNN/DailyMail
Base (Fast-dLLM)41.2 (1.0×)18.2 (1.0×)
Ours40.8 (1.1×)18.1 (1.4×)

As shown in the table, our model preserves out-of-domain generation quality while achieving 1.1× and 1.4× speedups, respectively. These moderate speedups are expected because such general-purpose generation tasks naturally involve less explicit long-CoT reasoning.

Q3: What is the width of the restricted canvas used to regenerate the fast-thinking segment? How is the truncation point sampled?

As stated in line 237 of our manuscript, the truncation points are randomly selected. Consequently, the remaining canvas size after truncation ranges between 128 and 512 tokens.


We hope these responses address your concerns. Please let us know if you have any further questions.

Dear Reviewer ZjpS,

Thank you once again for your time and constructive feedback. Guided by your suggestions, we have substantially strengthened the paper by adding the component-wise ablation of SGC and PASS, refining our discussion of the accuracy-efficiency trade-off, and clarifying the canvas and truncation protocols.

With less than 24-hours left in the discussion window, we would be very grateful to know whether our rebuttal has satisfactorily addressed your concerns. If any questions remain, we would be glad to discuss them further before the discussion closes.

All the best,

Authors of Submission 11654

AuthorsOpenReview ↗

discussion ddl

Dear Reviewer ZjpS,

Thank you once again for your time and constructive review of our paper.

With less than one hour remaining before the discussion window closes, we wanted to kindly follow up to see if our rebuttal and additional experiments have well addressed your concerns. We would be very grateful for any final thoughts or feedback you might have before the deadline.

Thank you very much for your time and consideration.

Best regards,

Authors of Submission 11654

Reviewer_d4ZJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BC-dLLM to reduce reasoning trajectory length and per-step denoising cost. SGC creates compact traces by retaining prefixes of longer trajectories and regenerating the rest within a smaller canvas, followed by conditional SFT. PASS estimates generation progress from EOS probabilities in the current block to dynamically shrink the suffix context window. Evaluations on three models and four mathematical reasoning benchmarks report high latency speedup.

Critical strengths
  • The paper addresses a practical problem by jointly reducing reasoning length and per-step computation. Its use of canvas size to construct compact training traces is well suited to dLLMs.

  • The experiments show promising gains. Compared with dParallel on LLaDA-8B-Instruct, the method improves GSM8K accuracy from 79.1% to 81.0% while reducing latency from 1.2 to 1.1 seconds, with improvements in both metrics on MATH-500.

  • The evaluation covers the LLaDA and Dream families, with ablations on compression, suffix sampling, masking ratios, window sizes, and general capability retention.

Critical weaknesses

Critical Weaknesses

  • The Contribution of SGC

    SGC's use of compressed CoT data and conditional SFT closely relates to existing approaches such as C3oT [1] and TokenSkip [2]. Its more distinctive design is to retain a prefix of a long reasoning trajectory and regenerate a compact continuation within a smaller remaining canvas.

    However, the paper lacks a direct comparison against a simpler alternative: generating responses from scratch on smaller canvases, retaining those with correct final answers, and fine-tuning using the resulting short trajectories. The GPT-Compress and LLMLingua-Compress experiments compare different compressors but do not establish the additional benefit of truncation and regeneration over this alternative. Consequently, the effectiveness of SGC as a whole is not sufficiently distinguished from the contribution of its specific data-construction mechanism.

  • PASS's Practical Latency Advantage over DPad

    DPad [3] already employs a sliding suffix window and distance-decay sampling. PASS's main additional design is to adjust the suffix budget using an EOS-based signal.

    Table 2 reports similar accuracy for DPad and PASS and lower FLOPs for PASS, but does not provide a corresponding latency comparison. Likewise, Figure 11 reports component-level compute reductions without isolating PASS's runtime contribution. Because PASS requires an additional forward pass for progress estimation according to the pseudocode, lower FLOPs alone do not establish end-to-end acceleration. The latency–accuracy trade-off obtained by replacing PASS with DPad therefore remains unclear.

Minor Weaknesses

  • The evaluation lacks a difficulty-stratified analysis of quality and computational savings.

    An important motivation is to reduce canvas-induced redundancy on problems that could be solved with shorter reasoning. However, aggregate accuracy, response length, and compute do not reveal which problems account for the savings or whether harder problems retain sufficient reasoning computation. Figure 6 groups examples by final response length, which is not an independent measure of difficulty. The experiments therefore do not directly substantiate the motivation that unnecessary computation is reduced on easier problems while necessary computation is preserved on harder ones.

  • The selection and evaluation protocol of Figure 3(a) limits the interpretation of its accuracy comparison.

    The authors select problems already answered correctly with a canvas size of 128, making the 100% accuracy at that setting directly related to the selection criterion. The paper does not clearly establish whether the plotted result at 128 comes from an independent reevaluation or sufficiently specify decoding randomness. Under stochastic decoding, reusing the selection outcomes while generating fresh responses at other canvas sizes would favor the 128 baseline. Under deterministic decoding, this concern would not apply, but the conclusion would still be restricted to the selected subset. Furthermore, describing accuracy as “essentially unchanged” does not adequately reflect the plotted decrease from 100% to approximately 88–93%.

  • Several experimental and implementation details are underspecified.

    Results do not explicitly identify the slow/fast mode. It is unclear whether the EOS-estimation forward pass in PASS is included in latency, step counts, and FLOPs. The concrete suffix-selection procedure inside PASS is not fully specified. In addition, interpreting Equation (5) with a standard sigmoid does not directly reproduce the early window sizes in Figure 10. These issues primarily affect reproducibility and interpretation.

[1] Kang et al. C3oT: Generating Shorter Chain-of-Thought Without Compromising Effectiveness. AAAI 2025

[2] Xia et al. TokenSkip: Controllable Chain-of-Thought Compression in LLMs. EMNLP 2025

[3] Chen et al. DPad: Efficient Diffusion Language Models with Suffix Dropout. ICLR 2026

Questions for authors

Additional Experiments

Q1. Does SGC's truncation-and-regeneration procedure outperform directly generating short trajectories?

Could the authors compare the following approaches using the same problem pool, training-set size, optimization settings, and inference configuration?

  • Generate responses from scratch on smaller canvases, retain correct trajectories, and fine-tune.

  • Generate long trajectories, truncate them, regenerate continuations within smaller canvases, retain correct trajectories, and fine-tune as in SGC.

Please report post-training accuracy, response length, decoding steps, and actual latency, together with the data-generation budgets and filtering acceptance rates.

As an optional extension, comparing fast-only training against conditional slow/fast training would help explain the additional value of the dual-mode design.

Q2. How much faster is PASS than DPad?

Could the authors compare the following configurations on the same SGC checkpoint?

  • No suffix sparsification.

  • DPad.

  • PASS.

Please report actual generation latency including all auxiliary operations.

Q3. Does acceleration compromise problems requiring long CoTs?

Could the authors evaluate whether SGC and SGC+PASS cause substantial accuracy degradation on problems that require long CoTs, where compression may remove necessary reasoning steps?

Clarifications

Q4. Which thinking mode and component configuration are used in experiments?

Please specify the slow/fast mode and PASS setting for Table 1, Figure 4, Figures 5(a–b), and Tables 2–4. For example, the LLaDA/GSM8K result in Table 1 matches the fast operating point in Figure 1, but other entries are not individually identified. Figure 4 specifies changes to the confidence threshold without explicitly identifying the thinking mode, while Table 4 leaves the prompting and acceleration configuration unclear.

Q5. How is the EOS-estimation forward pass accounted for and reused?

Is the full forward pass at the beginning of each block in Algorithm 1 included in latency, decoding steps, and FLOPs? Are its predictions reused for the first token update? Please specify the timing boundaries and, if possible, report the runtime fractions attributable to progress estimation, sparse-input construction, and within-block decoding.

Q6. How does Equation (5) correspond to the window sizes in Figure 10?

If σ\sigma is the standard sigmoid and Pbpg∈[0,1]P_b^{pg}\in[0,1], then:

Db∈[0.5Dinit,,0.731Dinit].D_b\in[0.5D^{\mathrm{init}},,0.731D^{\mathrm{init}}].

For Dinit=256D^{\mathrm{init}}=256, this gives approximately 128–187, whereas Figure 10 shows early values close to 256. Is there a special initialization, additional scaling, or a difference between the equation and implementation?

Q7. Could the authors specify the remaining implementation and evaluation details?

How does fPASSf_{\mathrm{PASS}} select individual suffix positions, round window sizes, preserve original positional encodings, and synchronize the sparse input with updates to the current block?

How are SGC's truncation points and remaining canvas budgets selected?

For Figure 3(a), please also specify the subset size, sampling temperature, random seeds, and whether the result at 128 reuses the selection outcomes.

5 comments
AuthorsOpenReview ↗

Author Responses (1/2)

We extend our sincere gratitude for your insightful reviews, which helps emphasize the contributions of our work. To the weaknesses and some questions, our response is as follows.

Critical-W1 & Q1: The Contribution of SGC — How does SGC's truncation-and-regeneration mechanism compare with simply fine-tuning on correct short trajectories generated from scratch on smaller canvases?

We thank the reviewer for this insightful comment. The core motivation behind SGC's truncation-and-regeneration design is to avoid restricting the training set solely to compressed or short reasoning trajectories. Instead, by retaining prefixes of varying lengths from long CoT trajectories and regenerating compact continuations, SGC constructs a hybrid training distribution that bridges long and short CoT reasoning.

Following the suggestion, we conducted a direct comparison against the simpler alternative (Short-response only), in which we generate responses on smaller canvases, retain the correct short trajectories, and fine-tune the model solely on them. Table 1 reports accuracy (%) / average decoding steps across different decoding confidence thresholds:

Table 1. Comparison between BC-dLLM (with SGC's truncation-and-regeneration mechanism) and fine-tuning on short responses only (accuracy / decoding steps).

Threshold0.950.900.800.700.60
BC-dLLM82.4/6481.0/3978.9/3074.5/2470.1/20
Short-response only81.9/5381.1/4879.2/3776.4/3171.9/26

As shown in the results, while training exclusively on short CoT trajectories (Short-response only) can also improve inference parallelism, it entails a less favorable trade-off between accuracy and efficiency and lowers the model's performance ceiling (from 82.4% to 81.9% at a threshold of 0.95). This demonstrates the advantage of SGC's data-construction mechanism over simply filtering for short responses.

Critical-W2 & Q2: PASS's Practical Latency Advantage over DPad

We thank the reviewer for this valuable feedback. Due to the temporary unavailability of H100 GPUs during the rebuttal period, we conducted the wall-clock latency comparison against DPad on NVIDIA H20 GPUs. Table 2 reports accuracy (%) / average latency across different decoding confidence thresholds:

Table 2. Comparison between DPad and BC-dLLM (with PASS) across confidence thresholds (accuracy / latency).

Threshold0.90.80.70.6
DPad80.9/4.978.3/3.373.7/2.970.0/2.6
BC-dLLM81.0/4.878.9/3.374.5/2.870.1/2.6

Compared with DPad, PASS yields a relatively modest reduction in inference latency but achieves noticeably better accuracy retention at the same or slightly lower latency (e.g., +0.6% at a threshold of 0.80 and +0.8% at a threshold of 0.70). This validates our core motivation: the amount of suffix context needed during dLLM decoding changes dynamically, and adopting an adaptive suffix length effectively improves overall inference efficiency without sacrificing reasoning quality.

Minor-W1 & Q3: The analysis does not reveal which problems account for the savings or whether harder problems retain sufficient reasoning computation.

Thank you for this constructive suggestion. To examine this, we divide the test set into four groups (G1 to G4, from shortest to longest) according to the response lengths of the original Base model and compare accuracy before and after compressing the reasoning trajectories:

Overall, for samples with relatively short to moderate responses (G1–G3), compressing the CoT results in minimal performance loss (and even slight improvements in G1 and G3). However, for tasks requiring highly complex reasoning and the longest trajectories (G4), the performance degradation caused by compression is more pronounced (68.6% vs. 63.7%). We will add this analysis by group to the revised paper to provide a clearer view of the trade-off between accuracy and efficiency across problem difficulties.

ModelG1G2G3G4AVG
Base model89.889.081.868.682.3
BC-dLLM90.787.182.563.781.0
AuthorsOpenReview ↗

Author Responses (2/2)

Minor-W2: The selection and evaluation protocol of Figure 3(a) limits its interpretation. Selecting problems already solved at a canvas size of 128 may introduce selection bias if decoding is stochastic, restricts conclusions to a filtered subset, and makes describing the drop from 100% to 88–93% as "essentially unchanged" imprecise.

We thank the reviewer for this detailed observation and the opportunity to clarify our evaluation protocol and core motivation in Figure 3(a):

  • In our inference experiments, we set rollout.temperature = 0.0 (deterministic greedy decoding). Therefore, there is no decoding randomness, and the concern that reusing selection outcomes might unfairly favor a canvas size of 128 does not apply.

  • Restricting Figure 3(a) to problems already solved at a canvas size of 128 is a deliberate experimental design rather than a limitation. Our primary goal here is to illustrate a specific behavioral phenomenon in dLLMs: as the canvas size increases, the model tends to generate unnecessarily long responses even for simple problems that can already be solved within a small canvas budget (128).

  • We fully recognize that on the full test set, a larger canvas naturally brings overall accuracy gains by enabling the model to solve more complex problems. To provide a complete picture, the table below shows how accuracy and average response length vary with canvas size on a randomly sampled subset of 300 test problems.

    Canvas size128192256320384512
    Accuracy (%)74.075.979.480.582.183.0
    Mean response length123188245287314328
  • However, this does not contradict our core motivation: dLLMs inherently produce unnecessarily long responses with larger canvases. This motivates us to design an effective algorithm that improves canvas utilization efficiency while largely preserving the model's reasoning performance.

Minor-W3 & Q7: Several experimental and implementation details are underspecified.

We thank the reviewer for requesting these details and clarify them point by point below:

  • PASS implementation details: For suffix positions, we select the contiguous suffix tokens immediately to the right of the current block being decoded. The window size is converted to an integer via standard rounding (round). Furthermore, because the selected suffix is contiguous with the current block, the resulting input remains a continuous sequence. Therefore, no adjustment to the positional encoding indices is needed, and token updates within the current block are directly synchronized in place.

  • SGC truncation and canvas budgets: As stated in line 237 of our manuscript, the truncation points are randomly selected. Consequently, the remaining canvas size after truncation ranges between 128 and 512 tokens.

  • Figure 3(a) evaluation setup: We evaluate on a subset of 300 problems with the sampling temperature set to temperature = 0.0 (deterministic decoding). Under deterministic decoding, re-evaluating at a canvas size of 128 is identical to reusing the selection outcomes, avoiding any stochastic bias.

Q4. Which thinking mode and component configuration are used in experiments?

Thank you for asking for this clarification. Unless otherwise stated, all experiments in our paper use the fast-thinking mode as the default setting.

Q5. How is the EOS-estimation forward pass accounted for and reused? Is the initial full forward pass per block included in the reported metrics (latency, steps, and FLOPs) and reused for the first token update? What is the runtime breakdown?

Yes. The full forward pass at the beginning of each block in Algorithm 1 is fully included in all reported metrics, including latency, decoding steps, and FLOPs. Furthermore, its predictions are directly reused to perform the first token update within that block, thereby avoiding any redundant forward computation.

Q6. How does Equation (5) correspond to the window sizes in Figure 10?

We thank the reviewer for this careful observation. You are correct: with a standard unscaled sigmoid function σ(x)\sigma(x), if the initial window size is 256, the upper bound of the window size would be 256×0.731256 \times 0.731, rather than 256.

In our implementation, to ensure that the dynamic window size can smoothly span the full range up to the initial budget, we scale the sigmoid input by a factor of 1010, using σ(10x)\sigma(10x). Since σ(10)≈1.0\sigma(10) \approx 1.0, the maximum window size closely approaches 256. We omitted this constant scaling factor in Equation (5) for notational simplicity, and we will explicitly include it in the revised manuscript to avoid confusion.


Hope that our response can well addresses your concerns. Please let us know if you have any new questions or concerns.

Thank you for the additional experiments and detailed clarifications. These results address several of my initial concerns and provide a clearer understanding of the method's benefits and trade-offs. I also appreciate the clarification regarding thinking modes and the accounting and reuse of the EOS-estimation forward pass.

My remaining concerns are:

  1. PASS's incremental practical benefit appears limited.
    The new results show latency reductions of approximately 0–3.4% relative to DPad, alongside accuracy improvements of 0.1–0.8 percentage points. These are positive but modest gains. Without repeated timing measurements or uncertainty estimates, the robustness of the small latency differences remains unclear. A no-sparsification baseline on the same SGC checkpoint would also help establish PASS's net latency benefit.

  2. The long-CoT analysis reveals a trade-off that may limit practical benefits.
    I appreciate the additional grouped analysis, which directly addresses my question and improves the characterization of the method. The longest baseline-CoT group shows a 4.9-point accuracy decrease, compared with a 1.3-point decrease overall. This larger loss suggests that the reported efficiency gains may offer less practical benefit for applications where maintaining accuracy on long reasoning trajectories is important. It would be helpful to acknowledge this trade-off explicitly when discussing the method's applicability.

Reviewer_PNC4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BC-dLLM to reduce redundancy arising from fixed-length canvases in discrete diffusion language models. Self-Guided CoT Compression (SGC) learns compact reasoning from self-generated trajectories, while Progress-Aware Suffix Sparsification (PASS) uses EOS probabilities to dynamically adjust the suffix context window. Experiments on three open-source dLLMs and four mathematical reasoning benchmarks, together with ablations, demonstrate favorable efficiency–performance trade-offs.

Critical strengths
  1. This paper examines how oversized canvases can also encourage unnecessarily long reasoning and redundant suffix processing, motivating joint optimization of trajectory length and context computation. The reported acceleration over Original at comparable task performance supports the practical value of this perspective.

  2. This paper presents two technical contributions: SGC and PASS. These components are complementary, and ablation studies support their effectiveness. The figures are well organized and visually appealing, helping readers understand the method and experimental results.

Critical weaknesses
  1. Incomplete evidence of advantages over strong baselines. Latency gains over dParallel are modest in some settings: approximately 0.1–0.2 seconds on LLaDA-8B-Instruct, although accuracy also improves. Table 1 and Figure 4(b) omit dParallel results on LLaDA-1.5, and the inference hardware and timing protocol are insufficiently specified. These gaps limit assessment of the method’s advantages across models and the comparability of the reported latencies.

  2. Limited validation of acceleration beyond mathematical reasoning. Table 4 reports task performance on MBPP, MMLU, and other general benchmarks, but not inference efficiency. The efficiency–performance trade-off of the complete framework on code generation and general question answering therefore remains unclear, limiting the demonstrated scope of its benefits.

Questions for authors
  1. Could the authors report dParallel’s accuracy, average decoding steps, and latency on LLaDA-1.5 under consistent evaluation conditions, and include its trade-off curve in Figure 4(b)?

  2. Could the authors compare the complete BC-dLLM framework with Original on MBPP and MMLU, specifying the evaluation protocol and reporting task performance, latency, and decoding steps where applicable? If acceleration is not expected in these settings, please clarify the intended scope.

  3. Section 4.1 specifies H20 GPUs for training. Could the authors clarify the hardware used for the inference latency measurements in Table 1, and confirm whether all methods were timed under the same hardware and measurement protocol?

1 comment

We thank the reviewer for the thoughtful comments and suggestions. Our responses are as follows.

W1 & Q1: Table 1 and Figure 4(b) omit dParallel results on LLaDA-1.5, and the inference hardware and timing protocol are insufficiently specified.

Thank you for this helpful comment.

  • Hardware and timing protocol: Unless otherwise specified, all inference latency measurements (including Table 1) are conducted on 8 NVIDIA H100 (80 GB) GPUs with a batch size of 8.
  • dParallel results on LLaDA-1.5: Because the authors of dParallel did not release a checkpoint for LLaDA-1.5, we reproduced the LLaDA-1.5 checkpoint based on their official codebase for a fair comparison. The results across confidence thresholds (accuracy (%) / decoding steps) are reported below:

Table 1. Comparison between BC-dLLM and the reproduced dParallel model on LLaDA-1.5 across confidence thresholds (accuracy / decoding steps).

Threshold0.900.800.700.60
BC-dLLM80.2/3477.9/2675.4/2168.0/18
dParallel81.0/3979.8/3274.2/2763.0/23

Across all thresholds, BC-dLLM consistently uses significantly fewer decoding steps than dParallel. Under matched step budgets, BC-dLLM achieves clear accuracy gains. For example, BC-dLLM at a threshold of 0.80 (77.9%/26 steps) outperforms dParallel at 0.70 (74.2%/27 steps) by +3.7%. At thresholds corresponding to greater acceleration (0.70 and 0.60), BC-dLLM outperforms dParallel in both accuracy (+1.2% and +5.0%) and step efficiency. We will add these results and hardware details to the revised manuscript.

W2 & Q2: Validation of acceleration beyond mathematical reasoning.

Thank you for these comments. Our primary motivation is to improve long-CoT generation efficiency while preserving the parallel decoding capability of dLLMs. Therefore, our main evaluation focuses on GSM8K and MATH-500. Most existing general-knowledge evaluations use answer-option log-likelihoods and do not measure free-form generation efficiency. We therefore add TruthfulQA-Gen and CNN/DailyMail. Our model preserves generation quality while achieving 1.1× and 1.4× speedups. The moderate gains are expected because these tasks involve less explicit long-CoT reasoning.

MethodTruthfulQA-GenCNN/DailyMail
Base (Fast-dLLM)41.2 (1.0×)18.2 (1.0×)
Ours40.8 (1.1×)18.1 (1.4×)

Q3: The hardware used for the inference latency measurements in Table 1.

Thank you for pointing this out. Unless otherwise specified, all inference latency measurements in our paper (including Table 1) are conducted on NVIDIA H100 GPUs. We will explicitly add these hardware details to the experimental setup section in the revised manuscript.


We hope these responses address your concerns. Please let us know if you have any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.