acceptodds
Under review as a conference paper at ICLR 2027

SLICE: Structural Self-Contrastive Guidance for Diffusion Language Models

Abstract

Diffusion language models (DLMs) are emerging as an alternative to autoregressive language models, generating text by iterative decoding. This paradigm offers flexibility in generation order and inference-time compute, yet effectively exploiting this flexibility to improve generation quality remains challenging. Existing approaches typically improve generation through additional training, external reference models, confidence calibration, self-contrastive guidance or computationally expensive multi-trajectory search and aggregation, leaving the model's internal computational structure underexplored for inference-time guidance. To this end, we propose Selective Layer Interventions for Contrastive Enhancement (SLICE), the first method to guide DLM decoding through structural self-contrast within a single model. SLICE exploits depth-wise redundancy through prompt-adaptive layer skipping to construct a weakened branch without additional training. It converts the resulting storng–weak logit discrepancy into a bounded structural contrast to reweight the predictive distribution at each decoding step. We characterize the guided distribution as an exponential reweighting and the unique optimizer of a KL-regularized objective, and bound its single-step deviation from the full-model prediction. Across eight benchmarks, SLICE outperforms the low-confidence baseline on both LLaDA-Instruct and LLaDA-1.5 under the same decoding setting, with average gains of 5.91 and 3.67 percentage points, respectively.

Primary area:
generative models
Keywords:
Diffusion language models, Self-contrastive guidance, Inference-time enhancement

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GFaqOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SLICE, a training-free decoding method for Diffusion Language Models (DLMs). SLICE constructs a structural self-contrastive signal by skipping redundant layers to form a "weak branch," which is then contrasted with the full model to guide decoding. The method is evaluated on LLaDA models across multiple benchmarks, showing solid performance improvements.

Critical strengths
  1. Solid Empirical Results: The experiments are comprehensive and rigorous. The proposed method achieves strong and consistent performance improvements across various benchmarks (understanding, math, coding) compared to the baselines.
  2. Application in DLMs: To the best of my knowledge, this is the first work to adapt the concepts of "layer contrast" and "layer redundancy" from autoregressive models into DLMs.
Critical weaknesses
  1. Limited Methodological Novelty: The core ideas of layer contrast and layer redundancy have been extensively explored in the autoregressive literature. Simply transplanting these existing concepts to DLMs, without fundamentally new mechanisms, makes the theoretical innovation somewhat incremental. I suggest the authors elaborate more on why this specific adaptation to DLMs is non-trivial compared to AR models.
  2. Incomplete Cost Analysis & Scaling Concerns: While the authors claim high computational efficiency, the cost analysis seems incomplete. Does the reported latency explicitly include the overhead of the initial forward pass for layer similarity probing, the layer pruning operations, and the additional GPU memory usage? More importantly, how does this overhead scale with longer sequence lengths? I hope the authors can provide a more holistic evaluation of the total cost to justify whether the performance gains are worth the computational trade-offs at scale.
  3. Lack of Evaluation on Advanced/Larger Models: Since the proposed method operates purely at the decoding stage and is training-free, it is highly encouraged to validate it on more advanced or larger DLMs. The current experiments are limited to 8B. Testing on newer models, such as the LLaDA 2.x series (especially the mini versions), would significantly strengthen the claim of the method's generalizability and robustness.
Questions for authors
  1. Clarify the novelty.
  2. Provide more cost analysis.
  3. Evaluate more advanced/larger models.
8 comments
AuthorsOpenReview ↗

Response to Reviewer GFaq (1/2)

We sincerely thank the reviewer for the constructive suggestions. We provide detailed explanations to address your concerns below.


W1/Q1: Limited Methodological Novelty.

We sincerely thank the reviewer for encouraging us to clarify how SLICE differs from prior AR methods and why its adaptation to DLMs requires specific design choices. SLICE contributes both a structural reference construction and a guidance mechanism tailored to masked diffusion decoding.

(a) Structural reference construction. SLICE uses layer similarity to select prompt-adaptive skips and constructs a shared-weight weak branch that retains downstream computation. This turns redundancy into a mechanism for generating contrastive guidance. In comparison, AR layer-contrast methods such as DoLa contrast intermediate- and final-layer predictions.

(b) Guidance tailored to masked diffusion decoding. In DLMs, guidance affects both token predictions and confidence-based position selection, changing subsequent contexts and decoding trajectories. SLICE evaluates both branches at the same partially masked state, independently standardizes their logits, then clips and rescales the contrast before token commitment. These operations address branch-scale differences and control the correction’s magnitude, making structural contrast usable within iterative, multi-position decoding. We have clarified DLM-specific guidance in Section 3.3 (lines 174–187) and the structural-reference rationale in Appendix A.3 (lines 835–841).

AuthorsOpenReview ↗

Response to Reviewer GFaq (2/2)


W2/Q2: Incomplete Cost Analysis & Scaling Concerns.

We sincerely thank the reviewer for this suggestion and acknowledge that we did not clearly explain how latency was measured or quantify the additional GPU memory usage.

(a) Cost analysis. We do not claim that SLICE is computationally cheaper than standard decoding: Table 2 explicitly reports generation latency of 1.68×/1.81× relative to Standard. Our efficiency claims refer to the accuracy–cost trade-off relative to other inference-enhancement methods and compatibility with acceleration. In particular, Table 5 shows that Fast-dLLM+SLICE improves six-task average accuracy from 33.91% to 38.20% at 0.99× Standard latency. The reported wall-clock generation time includes prompt-time probing, similarity computation, layer selection and bypass operations, both decoding branches, and logit guidance. We have made this explicit in Table 2’s caption (lines 353–354) and Appendix A.4 (lines 855–864).

(b) Scaling Concerns. Both branches share model weights and the additional memory mainly comes from temporary probing and guidance tensors. Table 3 reports accuracy gains across generation lengths, and we have added the cost measurements below in Appendix C.7, Table 15.

L / NStandard latency (s/example)SLICE latency (s/example)Standard peak allocated (GiB)SLICE peak allocated (GiB)
128 / 642.2543.78715.04716.252
256 / 1286.06910.43915.07716.716
512 / 25619.68034.43115.13717.240
1024 / 51271.330128.39415.26017.793

As generation length increases (128→1,024), SLICE peak memory grows only 1.095× (16.252→17.793 GiB). Thus, the additional memory grows much more slowly than the generation budget over this measured range. Relative latency remains 1.68–1.80× Standard. For tighter time budgets, Table 5 shows that Fast-dLLM+SLICE improves average accuracy from 33.91% to 38.20% at 0.99× Standard latency.


W3/Q3: Lack of Evaluation on Advanced/Larger Models.

We sincerely thank the reviewer for suggesting evaluations on newer and larger DLMs. We have evaluated SLICE on LLaDA2.0-mini, a newer MoE diffusion model with approximately 16B total parameters and 1.4B active parameters. SLICE improves ARC-C accuracy under both evaluated decoding budgets. We have added these results in Appendix C.6, Table 14:

B / L / NBenchmarkStandard (%)SLICE (%)Gain (pp)
32 / 128 / 64ARC-C23.1226.40+3.28
32 / 512 / 256ARC-C89.4292.36+2.94

These results demonstrate that SLICE also works on a newer MoE model with more total parameters than the original 8B checkpoints. For cross-family generalization, our response to Reviewer 44ks, W2(b), reports improvements on all three evaluated Dream benchmarks.


We hope the clarified methodological distinctions, additional cost measurements, and LLaDA2.0-mini results address the reviewer’s three main questions. We invite the reviewer to reconsider the assessment in light of these clarifications and results.

Thank you for the authors' efforts and detailed responses. Given the limited granularity of the rating scale, my current score is raised to 2.5. I would be willing to increase it to 3 if the following concerns are addressed. I encourage the AC to take this assessment into consideration.

  • Evaluation on LLaDA-2.0-mini: The current evaluation remains limited. Since SLICE is training-free and operates entirely at inference time, I encourage the authors to additionally evaluate it at least on MATH-500, HumanEval, and Countdown.

  • Layer selection: Please clarify whether the skipped layers are the same as those selected for ARC-C, to better assess the generalizability of the layer selection strategy.

  • Inference overhead: Please report the inference-time overhead on LLaDA-2.0-mini.

Reviewer_T2ZwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SLICE, a training-free decoding method for diffusion language models (DLMs). SLICE uses layer redundancy to construct a weakened branch through prompt-adaptive layer skipping and uses the logit differences between the full and weakened branches for contrastive guidance. The authors provide theoretical analysis of the guided distribution and its bounded deviation from the original predictions. Experiments on LLaDA models across eight benchmarks demonstrate performance improvements over baseline decoding methods.

Critical strengths
  1. The paper introduces a structural self-contrastive guidance method that uses layer redundancy in DLMs. The method is training-free and does not require external models.

  2. SLICE combines prompt-adaptive layer selection with normalized and bounded contrastive guidance. The method preserves the original decoding procedure, making it flexible and compatible with existing decoding and acceleration methods.

Critical weaknesses
  1. While the proposed method achieves improvements on standard benchmarks, its broader impact remains unclear. The evaluation is limited to two models from the same LLaDA family and commonly used benchmarks. Given the growing number of inference-time decoding methods for DLMs, it would be valuable to demonstrate how SLICE generalizes beyond these settings and offers broader practical benefits.

  2. The paper assumes that highly similar adjacent layers are suitable candidates for constructing a weakened branch. However, high representation similarity does not necessarily imply functional redundancy, nor does it guarantee that skipping these layers produces informative contrastive signals. The motivation behind this selection criterion could be better justified.

Questions for authors

Could the authors further discuss the broader applicability and practical significance of SLICE? In particular, how well is the method expected to generalize to other DLM architectures and more diverse real-world tasks beyond the two LLaDA models and standard benchmarks evaluated in this paper?

Could the authors provide further justification for the connection between layer redundancy and effective contrastive guidance? Specifically, why should high representation similarity indicate that certain layers can be skipped to construct an informative weak branch, and why should the resulting prediction discrepancies help correct errors in the full model rather than introduce arbitrary perturbations?

3 comments
AuthorsOpenReview ↗

Response to Reviewer T2Zw (1/2)

We sincerely thank the reviewer for the insightful comments on the underlying mechanism and the positive assessment of our work.


W1: Generalizability and practical benefits beyond current settings.

We sincerely thank the reviewer for highlighting the need to demonstrate SLICE’s broader applicability and practical benefits beyond the evaluated models and benchmarks. SLICE could extend to another model family, full-sequence decoding, and a real-world planning task Trip Plan. It also combines with existing acceleration and inference-enhancement methods, providing additional accuracy–latency choices.

(a) Cross-model generalization. On Dream-Instruct-7B, which belongs to a different family from LLaDA, SLICE improves performance across all three evaluated benchmarks:

BenchmarkStandardSLICEGain (pp)
GSM8K81.8884.70+2.82
HumanEval56.1058.54+2.44
MATH50040.8042.80+2.00

Under full-sequence decoding, SLICE also achieves larger gains over the corresponding unguided baselines, as reported in our response to Reviewer 44ks.

(b) Broader task applicability. We additionally evaluate SLICE on Trip Plan, a real-world-inspired task requiring itineraries to satisfy multiple planning constraints. SLICE increases accuracy from 10.63% to 11.44%, extending the evaluation beyond the original task types. We have added this evaluation in Appendix C.5, Table 13.

BenchmarkStandard (%)SLICE (%)Gain (pp)
Trip Plan10.6311.44+0.81

Q1: Broader applicability and practical significance of SLICE.

As reported in W1, SLICE improves performance across all three evaluated Dream benchmarks and on Trip Plan, a real-world-inspired planning task. These results extend the evidence to another model family and a task requiring multi-constraint planning, beyond the original benchmarks.

AuthorsOpenReview ↗

Response to Reviewer T2Zw (2/2)


W2: Justification for similarity-based layer selection.

We sincerely thank the reviewer for this insightful comment. Our intuition is that skipping highly similar layers retains much of the full model’s predictive behavior while removing part of its computation. The resulting full–weak contrast may therefore highlight predictions that receive additional support from the full computation. Specifically, Figure 2 and Table 8 in Appendix A.3 show that similarity-based skipping yields lower KL divergence and logit distance from the full model than random skipping. Table 4 further shows that SLICE outperforms random-skip guidance and weak-only decoding in all 12 comparisons. These results support the selection criterion empirically, without implying that layer similarity alone guarantees beneficial guidance. We have clarified this rationale in Section 3.2 (lines 167–171) and Appendix A.3 (lines 835–841).


Q2: Why does similarity-based layer skipping yield informative guidance?

As explained in W2, similarity guides the construction of a comparable weak reference, Appendix B.5 specifies a condition for beneficial guidance, and the ablations in Table 4 support its empirical effectiveness.


We appreciate the reviewer’s positive assessment of the structural contrastive construction. We hope the additional model and task evaluations with the clarified selection rationale, address the remaining questions about its broader applicability and mechanism.

Thank you again for your constructive review. We have added evaluations on Dream and Trip Plan and clarified why SLICE uses layer similarity to construct the weak branch. As the discussion period draws to a close, we would appreciate hearing whether these additions address your concerns. We would be happy to clarify any remaining concerns.

Reviewer_44ksOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SLICE, a training-free decoding method for masked diffusion language models. A prompt-only forward pass identifies layers with high input-output representation similarity and selects nonadjacent layers to bypass, creating a weak branch that shares the full model's weights. At each decoding step, both branches process the same sequence state. Their logits are standardized separately, contrasted, clipped, and rescaled to correct the full-model logits before token commitment. The paper characterizes this update as exponential reweighting and KL-regularized optimization, with fixed-state deviation bounds and conditional token-probability improvement results.

Under block-wise decoding with B = 32 and L = 256, SLICE raises the eight-benchmark average from 45.33 to 51.24 on LLaDA-8B-Instruct and from 48.07 to 51.74 on LLaDA-8B-1.5. OTS, the strongest competing method by average score, achieves 47.61 and 49.33, respectively. SLICE incurs 1.68x/1.81x measured latency relative to low-confidence decoding. Component ablations, evaluations across five block-wise configurations, and integrations with KLASS and Fast-dLLM's parallel sampler further examine its effectiveness and cost.

Critical strengths
  1. The component comparisons support using structural degradation as a contrastive reference. Both branches share pretrained weights, with no parameter updates or external scoring model. Table 4 distinguishes weak-only decoding from contrastive use of the weak branch and includes random-skip guidance averaged over three runs. For LLaDA-Instruct at B = 64, L = 512, the three-task averages are 59.32 for Base, 8.01 for weak-only decoding, 62.27 for random-skip guidance, and 70.36 for SLICE. SLICE leads all 12 individual component comparisons. These results support the proposed contrastive construction and indicate that the choice of skipped layers matters. Appendix A.3 also compares weak branches at shared decoding states and matched realized skip counts, avoiding trajectory differences in the proximity diagnostic.

  2. The experiments cover several task types and report the additional inference cost transparently. Table 1 shows gains over low-confidence decoding on all eight tasks for both LLaDA variants. The paper explicitly states that the main comparison is not compute-matched and reports SLICE's 1.68x/1.81x latency. Table 5 provides a useful integration result: Fast-dLLM + SLICE reaches a six-task average of 38.20 at 0.99x baseline latency, compared with 33.91 at 1.00x for Base. Relative to Fast-dLLM alone, it improves from 36.65 to 38.20 while increasing latency from 0.70x to 0.99x. Appendix A.2 specifies that this uses the non-cache parallel sampler. These results support compatibility with acceleration and make the accuracy-cost tradeoff concrete, subject to the selection-protocol concern below.

Critical weaknesses
  1. The independence of configuration selection and final evaluation is unresolved. Appendix A states that configurations are selected separately for each model, task, and decoding setting, but gives final k/g values without specifying the candidate search, selection criterion, or validation split. Appendix A.1 explicitly says that the preliminary 256-example GPQA and Sudoku "selection subsets" are included in the final sets of 448 and 500 examples, respectively. If performance on these subsets informed configuration selection, the final scores include selection examples and may overestimate generalization. The overlap is documented; its precise use and the magnitude of any resulting bias are not established. Selection procedures for the other tasks and tuning details for competing methods are also insufficiently specified. Consequently, the reported gains and comparative ranking need verification under a documented independent selection protocol. The 77/80 result supports effectiveness under separately selected configurations, but does not by itself establish robustness of one fixed configuration across decoding settings.

  2. The practical advantage and generality of structural guidance are not sufficiently established. The comparison omits some related works, for instance, CCD (Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models, [ICML 2026]). CCD aggregates historical predictions without additional model forward passes, and its adaptive variant reduces decoding steps. It evaluates LLaDA-8B-Instruct on overlapping tasks, including GSM8K and HumanEval, making it a directly relevant baseline for SLICE's 1.68x/1.81x latency overhead. The omission leaves the incremental accuracy-cost benefit of the weak branch unresolved. Structural contrast remains a distinct mechanism; CCD's existence alone does not negate that contribution, and results from the two papers' different protocols cannot establish which method is superior.

    The evaluation also covers only two closely related LLaDA checkpoints. Since the method depends on layer redundancy and the predictive effect of bypassing layers, transfer to another family such as Dream is particularly relevant. The authors acknowledge the model-family limitation in Section 4.3. In addition, all five Table 3 settings have B < L, and the accelerator experiments also use B = 32, L = 256. These settings permit reordering within blocks but do not test full-sequence any-order decoding with B = L. Block constraints change the available contexts and commitment choices, so the existing results do not establish whether the benefits persist without those constraints. A focused evaluation on another family and under full-sequence decoding would address this gap.

Questions for authors
  1. What decisions used the preliminary GPQA and Sudoku subsets, and did their correctness scores influence k, g, or other design choices? Please specify the selection data, candidate ranges, criterion, and number of tried configurations for SLICE, together with the tuning procedure for the main competing methods, particularly OTS and FoCore. The concern is outcome-based configuration selection; computing an individual prompt's skip set from its hidden states is part of the stated inference algorithm.

  2. If the preliminary subsets were used for performance-based selection, can you report SLICE and the principal baselines on examples that played no role in any configuration or method selection, with all choices frozen? GPQA's remaining 192 and Sudoku's remaining 244 examples would be informative only if they were not also used in later selection. Otherwise, a fresh held-out evaluation would be needed. Paired problem-level confidence intervals on the score differences would help assess the resulting uncertainty. Please also clarify how selection and evaluation were separated for the other six tasks.

  3. How does SLICE compare with CCD on representative overlapping tasks using a common LLaDA-8B-Instruct checkpoint, prompts, evaluator, and independent validation-based tuning? A comparison with fixed-budget CCD under a common nominal decoding schedule, supplemented by an accuracy-latency comparison including CCD-DS, would test the benefit of additional weak-branch computation. Please measure actual wall-clock latency, including method-specific overhead: the public CCD paper primarily reports step-count reductions, which cannot be directly compared with SLICE's measured latency ratios.

  4. Is there a methodological limitation preventing evaluation on Dream or under full-sequence any-order decoding? A representative task subset would suffice to test whether the gains extend to these settings. For the block-wise versus full-sequence comparison, please hold generation length and nominal denoising steps fixed, include the corresponding unguided baselines, select hyperparameters independently of evaluation, and report actual latency. This would help separate an effect specific to LLaDA or block-constrained decoding from a more general benefit of structural guidance.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 44ks (1/2)

We sincerely thank the reviewer for the constructive suggestions, which help strengthen the clarity and completeness supporting our paper. Below, we provide detailed explanations to address your concerns.


W1: Unclear independence between configuration selection and final evaluation.

We sincerely thank the reviewer for this suggestion and acknowledge that we did not clearly explain the independence between parameter selection and evaluation in the paper. We clarify the selection procedure and report results on both the selection subsets and the remaining evaluation examples below.

(a) Configuration selection. For each model, task, and decoding setting, the initial search comprised 32 configurations, combining four skip budgets (k = 2, 4, 6, 8) with eight guidance scales (g = 0.2, 0.25, 0.35, 0.4, 0.5, 0.55, 0.75, 1.0). Configurations were selected by maximizing the score on the selection subset: 128 examples for HumanEval and 256 examples for each remaining benchmark. Throughout the search, all shared settings remained fixed as specified in Appendix A.

(b) Performance beyond the selection subsets. Using the same configurations as Table 1, we report results on the remaining examples of six benchmarks. These examples were not used for parameter selection. Each block reports Base → SLICE accuracy (%) for LLaDA-8B-Instruct at B=32, L=256, N=128.

TaskSelection subset (%)Remaining examples (%)
ARC-C73.05 → 87.1176.20 → 83.19
ARC-E83.20 → 96.4883.73 → 92.92
GSM8K79.30 → 84.3879.12 → 80.62
MATH50033.98 → 37.5031.97 → 37.30
Sudoku9.77 → 15.6311.07 → 15.57
GPQA20.70→ 28.5225.52 → 30.73

SLICE improves accuracy on the remaining evaluation examples across all six tasks, with gains of approximately 1.5–9.2 percentage points, showing that the improvements extend beyond the selection examples. We hope these details clarify our procedure and provide further evidence of generalization beyond the selection subsets. We have documented the selection protocol in Section 4.1 (lines 299–300) and Appendix A (lines 758–766), with separate split results in Appendix C.1, Table 9.


Q1: Detailed configuration-selection procedure.

As detailed in W1(a), selection-subset scores were used only to choose k and g and all other hyperparameters remained fixed. We adopted competing methods’ configurations from their official releases without additional local tuning. OTS uses five beams and temperature 0.4. FoCore uses guidance scale 0.3, EMA decay 0.9, vocabulary truncation size 256, and eight negative-mask positions. These baseline configurations are now documented in Appendix A (lines 767–773).


Q2: Independence of final evaluation from configuration selection.

As reported in W1, configurations were frozen before evaluating the remaining examples, which played no role in configuration or method selection. The reported gains on these remaining examples support generalization beyond the selection subsets. We followed the same selection and evaluation procedure for the other tasks.

AuthorsOpenReview ↗

Response to Reviewer 44ks (2/2)


W2: Unclear practical benefits and generalizability of structural guidance.

We sincerely thank the reviewer for pointing out the omission of the relevant concurrent work CCD [1] and suggesting evaluations on another model family and under full-sequence decoding. Our additional experiments show that SLICE could easily transfer to Dream with improved accuracy, retain its gains without block constraints, and be combined with CCD-based decoding.

(a) Accuracy and efficiency compared with CCD. We evaluate SLICE, CCD-based methods, and their combinations under matched conditions (LLaDA-8B-Instruct, B=32, L=256 and N=128), using the official CCD implementation. We have added these comparisons in Appendix C.2, Table 10.

BenchmarkMetricStandardSLICECCDCCD-DSSLICE+CCD-DSSLICE+CCD
GSM8KAccuracy (%)79.1581.3579.6179.3880.2181.20
Latency (s/example)7.78713.0937.7266.96012.18113.193
HumanEvalPass@1 (%)37.8040.2438.4137.8039.0139.61
Latency (s/example)9.68318.0459.7186.51015.85818.153

SLICE achieves the highest accuracy on both benchmarks, while CCD-DS provides the lowest latency. Combining SLICE with CCD-DS improves accuracy over CCD-DS and reduces latency relative to SLICE alone, providing an additional accuracy–latency trade-off for different inference budgets.

(b) Transfer to Dream. We transfer the LLaDA parameters directly to Dream-Instruct-7B without Dream-specific parameter search. We have added the results in Appendix C.3, Table 11. Under matched evaluation conditions, SLICE improves accuracy on all three benchmarks. These gains support the transferability of the adopted configurations and the applicability of structural guidance beyond the LLaDA family.

BenchmarkStandard (%)SLICE (%)Gain (pp)
GSM8K81.8884.70+2.82
HumanEval56.1058.54+2.44
MATH50040.8042.80+2.00

(c) Full-sequence decoding. We hold L=128 and N=64 fixed and compare B=32 with B=L=128 using the same SLICE configurations. We have added the results in Appendix C.4, Table 12. SLICE improves full-sequence accuracy by 12.12 and 12.50 on ARC-C and ARC-E, respectively. Its absolute accuracy also exceeds its block-wise results on both tasks, showing that its effectiveness extends to full-sequence decoding.

TaskB=32, Standard → SLICE (%)B=L=128, Standard → SLICE (%)Full-sequence gain (pp)
ARC-C82.17 → 84.4774.15 → 86.26+12.12
ARC-E92.39 → 93.3182.37 → 94.87+12.50

[1] Kecheng Chen et al. Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models. ICML, 2026.


Q3: Comparison with CCD under common settings and measured latency.

As reported in W2(a), under matched settings and measured wall-clock latency, SLICE achieves the highest accuracy on both benchmarks, while CCD-DS is fastest. Their combination improves accuracy over CCD-DS while remaining faster than SLICE alone, offering an additional accuracy–latency trade-off.


Q4: Generalizability across models and decoding settings.

As reported in W2(b, c), SLICE improves performance on Dream by directly transferring the LLaDA configurations without Dream-specific tuning. It also achieves gains of 12.12 and 12.50 on ARC-C and ARC-E under full-sequence decoding, supporting its effectiveness across model families and decoding settings.


We thank the reviewer again for the detailed review. We hope the clarified selection process and additional comparisons address the main questions about evaluation independence, practical value, and generalizability, and invite the reviewer to reconsider the assessment in light of these results.

Thank you for the additional experiments and clarifications, which address several of my concerns. Some concerns remain regarding the comparative evaluation, so I maintain my original rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive suggestions and recognizing the strengths of our paper. The reviews highlight the novelty of SLICE as the first method to combine layer contrast and layer redundancy for inference enhancement in diffusion language models (GFaq). The reviews praise the comprehensive evaluation and consistent improvements across tasks (44ks, GFaq). The ablations support the choice of skipped layers (44ks), and the integration with existing methods highlights the method’s practical flexibility (T2Zw, 44ks).

The discussion has helped us strengthen both the methodological explanation and the empirical evidence. We summarize the main additions and clarifications below.

1. Configuration selection and evaluation independence (44ks W1/Q1–Q2)

  • We document the configuration search, the selection criterion and subsets, and the sources of baseline configurations.
  • With configurations fixed, SLICE improves over Standard by 1.5–9.2 percentage points on the remaining examples across all six benchmarks reported in the split analysis.

2. Generalization across models, tasks, and decoding settings (44ks W2/Q4, T2Zw W1/Q1, GFaq W3/Q3 and follow-up)

  • Dream and new tasks. Without Dream-specific tuning, SLICE improves all three Dream benchmarks by 2.00–2.82 points. It also improves the Trip Plan score from 10.63% to 11.44%.
  • Full-sequence decoding. With generation length and denoising steps fixed, SLICE improves over Standard by 12.12 points on ARC-C and 12.50 points on ARC-E without block constraints.
  • LLaDA-2.0-mini. With a shared 8,192-token generation limit, Standard reaches 84.20% on MATH500 and 85.37% on HumanEval, close to the official scores. SLICE improves over Standard by 1.00–1.56 points across MATH500, HumanEval, and Countdown, with fewer output tokens on average on each task.

3. Comparative accuracy and inference cost (44ks W2/Q3, GFaq W2/Q2)

  • CCD comparison. Under matched conditions on GSM8K and HumanEval, SLICE achieves the highest scores and CCD-DS has the lowest measured latency. Combining them improves scores over CCD-DS and reduces latency relative to SLICE alone.
  • Inference cost. We clarify that generation timing includes the initial probe, layer selection, both decoding branches, and guidance. We also report wall-clock latency and peak allocated GPU memory across generation budgets.

The main changes in the revision are:

  • Method and mechanism (Sections 3.2–3.3 and Appendix A.3). We explain how prompt-based layer skipping constructs a reference branch with shared weights and how SLICE applies the contrast at the same partially masked state.
  • Selection and measurement protocols (Section 4.1 and Appendix A). We document configuration selection, baseline configurations, and what is included in generation timing.
  • Additional experiments (Appendices C.1–C.7). We provide the split evaluation, CCD comparisons, transfer and full-sequence results, Trip Plan and Mini evaluations, and additional latency and memory measurements.

We thank the reviewers and the Area Chair for considering the revision and the evidence added during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.