acceptodds
Under review as a conference paper at ICLR 2027

Match the Canvas, Not Just the Answer: Predictability-Aware SFT for Diffusion Language Models

Abstract

Diffusion language models (DLMs) enable flexible parallel generation by resolving tokens according to the model's evolving predictability rather than a fixed left-to-right order. This flexibility, however, introduces training choices absent from standard autoregressive teacher forcing: which reference tokens are visible when a target is supervised and how much generation space is allocated. Standard supervised fine-tuning (SFT) typically makes these choices through random masking, independently of inference. Our analysis shows how this training–inference mismatch can misdirect supervision. Random masking can conceal prediction errors or penalize uncertainty that decoding would otherwise resolve, while the attention-visible input length can change which response formulations the model favors, even when the underlying solution is unchanged. Motivated by these findings, we propose Predictability-Aware Canvas Selection and Supervision (PACS), an SFT method. PACS uses the model's own predictability to select each example's attention-visible input length and supervises reference tokens immediately before their confidence-guided revelation. Across reasoning and code-generation benchmarks, PACS consistently achieves the highest mean accuracy among the compared SFT methods, with gains of up to 12.84 percentage points over vanilla SFT on individual benchmarks. (Code is available at https://anonymous.4open.science/r/pacs-FB4E/README.md.)

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Models, supervised fine-tunning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dp5zOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the training-inference mismatch in SFT for Diffusion Language Models (DLMs), where random masking and improper canvas extent can misdirect supervision. To resolve this, the authors propose PACS, a predictability-aware method that selects the optimal canvas length and supervises reference tokens immediately prior to confidence-guided revelation. Extensive experiments demonstrate that PACS outperforms standard SFT baselines across several reasoning and code generation benchmarks.

Critical strengths

The core idea is insightful and well-motivated, bringing a novel decoding-aware perspective to canvas construction during DLM training. Furthermore, the proposed method achieves noticeable performance improvements across multiple benchmarks.

Critical weaknesses
  1. Limited Experimental Scale and Generalizability. The current empirical evaluation is relatively light. The experiments should be expanded to include additional datasets and a broader range of modern DLM architectures/backbones (e.g., LLaDA 1.5, LLaDA 2.x, or larger-scale models). Without these, it is difficult to assess whether the proposed method can effectively scale or hold up across diverse base models.
  2. High Search Overhead and Scaling Bottleneck in Long-Context and Large-Scale Scenarios. The method relies on predictability-aware candidate search to select the optimal canvas. However, in the context of long-context scenarios and large-scale training data, the computational and token-level overhead for constructing, searching, and evaluating multiple candidate windows will explode. This prohibitive search and preprocessing overhead makes it extremely difficult for the approach to scale effectively to industrial-grade datasets and long sequences.
  3. New Latent Training-Inference Mismatch via Flawed Model Priors. The fundamental assumption behind PACS can still introduce latent training-inference mismatch. Prior to SFT, the model's pre-existing distribution dictates its prediction confidence, which may diverge from the ground truth. For instance, the model might assign a high confidence (0.80.8) to token AA at a position where the gold label is BB. Relying on the model's confidence to select canvases and trajectories can be flawed, as it trusts the model's "confidently wrong" belief to shape the supervision context.
Questions for authors

Same as "Critical Weaknesses"

4 comments

We sincerely thank reviewer for the thoughtful and constructive review. We appreciate the reviewer's concerns and the opportunity to further clarify our contributions.

To the best of our knowledge, our work is among the first to offer a unified perspective on DLM SFT by jointly considering reference predictability, training-canvas construction (which shapes learnability), and inference-time decoding behavior. Beyond the empirical improvements, we hope this conceptual perspective can inspire further research on more effective and better-aligned training methods for DLMs.

Below, we address each question and concern in detail. All the comments help us further clarify our work, and the reviewer's questions highlight some of the key distinctions of our method. We also appreciate that none of the points raised indicate fundamental flaws in our core ideas or methodology.

Weakness 1: Experimental scale and generalizability

We thank the reviewer for highlighting the need for broader evaluation. We have expanded our experiments to two additional models, LLaDA-1.5 and SDAR-8B-chat-32b, and two additional benchmarks, LiveCodeBench and GPQA. These experiments complement the LLaDA-8B-Instruct and Dream-7B-Instruct results in the submission. LLaDA-1.5 tests whether the gains persist across checkpoints within the LLaDA family, while SDAR extends the evaluation to a block-diffusion architecture. The additional benchmarks broaden coverage to contest programming and scientific reasoning.

LLaDA-1.5

MethodGSM8KMATHHumanEvalMBPPLiveCodeBenchGPQA
LLaDA-1.579.68%36.00%42.07%46.30%19.25%27.27%
Naive SFT79.83%36.80%45.73%49.03%22.98%29.29%
Semi-AR SFT79.98%37.20%47.56%51.75%24.98%31.31%
PACS80.36%38.60%51.22%54.86%27.98%33.33%

SDAR

MethodGSM8KMATHHumanEvalMBPPLiveCodeBenchGPQA
SDAR86.13%45.00%58.54%70.82%41.93%29.80%
Naive SFT87.04%46.00%59.76%71.98%41.30%31.31%
Semi-AR SFT87.57%45.80%60.98%71.98%43.47%33.84%
PACS88.32%48.60%63.41%75.48%45.96%35.86%

PACS outperforms both naive and semi-AR SFT on all six benchmarks for both additional models. These consistent gains strengthen the evidence that its benefits extend across initial checkpoints, model architectures, and evaluation tasks, including both newly added benchmarks. We will incorporate these experiments and their evaluation protocols into the revised paper.

Weakness 2: High search overhead and scaling bottleneck

We thank the reviewer for raising this important practical concern. While the main paper focuses on clearly presenting our novel conceptual contribution of illustrating how standard SFT can misdirect supervision in DLMs, we recognize the importance of computational efficiency in practice. PACS adds a bounded, adjustable computational multiplier over baseline methods, preserving its asymptotic scaling with sequence length. Below, we provide additional results demonstrating substantially improved efficiency, with accuracy that matches or exceeds best baseline at comparable training cost. Additional computation is an optional user choice for pursuing further accuracy gains, allowing practitioners to select a configuration that fits their computational budget.

  1. To the best of our knowledge, PACS's canvas-length search addresses a major source of training–inference mismatch overlooked by all existing SFT methods for DLMs; therefore, we further highlight its novel and significant conceptual contribution.

We view canvas-length search as a flexible user option rather than a fundamental operational bottleneck. Each additional level of refinement in the canvas-length search yields further improvements in training performance (Fig. 6), demonstrating the value of more precise canvas selection while allowing users to balance their computational resources. For example, users with limited training time can choose a coarser search granularity.

In contrast, the commonly used batch-max or dataset-max canvas lengths show clear disadvantages compared with predictability-aware canvas selection (Figs. 5 and B.1). Importantly, these benefits are not limited to PACS: simple canvas-length selection also improves baseline methods (Fig. B.1), highlighting the broader significance of this overlooked issue.

  1. Rather than viewing PACS as unnecessarily expensive or inherently difficult to scale, we hope to emphasize that its higher computational cost per training pass does not necessarily translate into higher overall training cost. Instead, its computation is directed toward addressing a broader, fundamental challenge: the training–inference mismatch in DLMs, which makes effective supervision difficult.

While other SFT methods (e.g., vanilla SFT) are computationally cheaper per training pass, they extract less effective learning signal from the same data and tend to reach a performance ceiling even with prolonged training. Therefore, we emphasize comparisons on the same dataset under matched training time (Appendix Table B.4). PACS continuesly improves with further training, whereas the baselines largely saturate, pushing the achievable accuracy frontier higher.

  1. To address the reviewer's valuable practical concern regarding training cost, we further offer an additional option to substantially reduce training cost of PACS while retaining competitive accuracy. We view the training confidence threshold as a flexible user choice for balancing computational cost and performance. This configuration substantially reduces training cost while maintaining competitive performance, as shown below:
ConfigurationTraining ThresholdTraining TimeHumanEvalMBPP
Semi-AR block SFT—≈1 h 16 min43.90%48.60%
PACS (partial run)0.2≈1 h 05 min44.51%48.60%
PACS (full run)0.2≈3 h 15 min45.73%49.01%
PACS (full run)0.9≈14 h 55 min47.56%49.42%

The partial run matches or exceeds best baseline on both benchmarks with less training time. Further training improves both scores, while the full run at threshold 0.2 remains approximately 4.6× faster than the original configuration. These results show that PACS can achieve competitive accuracy at a training cost comparable to semi-AR, and benefit from additional training in the evaluated setting while the baselines largely saturates.

Weakness 3: Training inference mismatch via flawed model prior

We sincerely thank the reviewer for raising this concern, which gives us an opportunity to clarify an important distinction in PACS. PACS's trajectory-guided supervision is designed to correct errors arising from the model's existing predictions, rather than reinforce potentially flawed model priors. In particular, model confidence guides where to supervise, not which predictions to trust.

In the reviewer’s example, confidently wrong predictions are precisely a case PACS is designed to correct. When the model confidently predicts A while the reference is B, PACS applies the loss −log⁡pθ(B∣C)-\log p_\theta(B \mid C) at the pre-commitment state, with B as the gold-label target. The low probability assigned to B produces a high reference loss precisely where the model would make the mistake of predicting A. The incorrect prediction A is never accepted as the training prefix.

However standard SFT can misdirect supervision because its masking pattern need not follow the model's decoding trajectory(Section 4.1, Figure 3).

We sincerely appreciate the reviewer's thoughtful feedback, which touches on the central observations and contributions of our work, and would be happy to provide further clarification or discuss any remaining questions.

Dear Reviewer dp5z,

We are sincerely grateful for your thoughtful feedback, which has given us another opportunity to reflect on our work, a process that has been both challenging and rewarding. We truly appreciate your engagement and would be grateful if you could let us know whether our rebuttal has adequately addressed your concerns.

Reviewer_skM9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how the training canvas affects SFT of masked diffusion language models: both which reference tokens are visible and how much attention-visible space is allocated. PACS selects a canvas using reference predictability along a confidence-guided trajectory, then supervises each token just before it is revealed. PACS has the highest average accuracy among the compared SFT methods at the evaluated tasks, especially on coding tasks.

Critical strengths
  • The method is intuitive, easy to understand, and the results are good to some extent.
Critical weaknesses

The main concern is that the return on the extra computation is modest compared to semi-AR (semi-AR is more like block diffusion if I understand correctly). The full OpenCoder run takes about 14 h 55 min on one H200, versus 1 h 13 min for vanilla and 1 h 16 min for semi-AR. However, semi-AR leads both tasks at four GPU-hours and HumanEval at eight; PACS leads both at twelve by only 0.40 points on MATH500 and 2.44 on HumanEval, equivalent to two and four net solved problems. The practical case for the more complex procedure therefore rests on a small gain in a larger-budget regime, with limited evidence that the trade-off is stable or generalizes to larger corpora.

Questions for authors
  1. Offline trajectory drift and iterative refresh. Since the trajectories are collected from a frozen initial model, I'm curious how much the reveal order and preferred canvas length change as fine-tuning progresses. Have you measured this drift or compared a fixed offline dataset with trajectories refreshed once or periodically during training?

  2. Potential inference speedup. I'm curious whether this supervision also makes confidence-threshold decoding more efficient. Does the fine-tuned model commit more correct tokens per forward pass and need fewer denoising steps?

  3. Training stage and initial model quality. Is PACS intended primarily for post-training, or could it also help during continued pretraining or earlier stages of learning? Collecting reference-conditioned trajectories across candidate lengths is costly, and a weaker model's confidence may provide a less useful reveal order. How sensitive are the gains and collection costs to the initial checkpoint's capability and confidence calibration? A comparison across initial checkpoints, with and without iterative refresh, would help establish when this approach becomes worthwhile.

  4. Not sure if the method still works on native block diffusion models like LLaDA-2/2.1-mini or SDAR.

5 comments

We sincerely thank reviewer for the thoughtful and constructive review. We appreciate the reviewer's concern and agree that accuracy improvements should be evaluated alongside the total post-training cost. We are especially grateful that the reviewer recognized the contributions of our work.

To the best of our knowledge, our work is among the first to offer a unified perspective on DLM SFT, by jointly considering reference predictability, training-canvas construction(which determines learnability), and inference-time decoding behavior. Beyond the empirical improvements, we hope this conceptual perspective can inspire further research on more effective and better-aligned training methods for DLMs.

Below, we address each question and concern in detail. All the comments are very constructive and help strengthen our work. We also appreciate that none of the points raised indicate fundamental flaws in our core ideas or methodology.

Weakness: High Computational Cost

We thank the reviewer for raising this important practical concern while also recognizing PACS's ability to achieve the highest accuracy with prolonged training. While the main paper focuses on clearly presenting our conceptual contribution by illustrating how standard SFT can misdirect supervision in DLMs, we recognize the importance of computational efficiency in practice. For simplicity, our original experiments use a confidence threshold of 0.9, matching the evaluation decoder.

To address the reviewer's valuable practical concern regarding training cost, we further offer an additional option to substantially reduce training cost of PACS while retaining competitive accuracy. We view the training confidence threshold as a flexible user choice for balancing computational cost and performance. This configuration substantially reduces training cost while maintaining competitive performance, as shown below:

ConfigurationTraining ThresholdTraining TimeHumanEvalMBPP
Semi-AR block SFT (full run)—≈1 h 16 min43.90%48.60%
PACS (partial run)0.2≈1 h 05 min44.51%48.60%
PACS (full run)0.2≈3 h 15 min45.73%49.01%
PACS (full run)0.9≈14 h 55 min47.56%49.42%

At matched training time, PACS achieves slightly higher accuracy than semi-AR block SFT. Extending the lower-threshold PACS run improves both scores further. These results show that PACS can achieve competitive accuracy at a training cost comparable to semi-AR, and benefit from additional training in the evaluated setting while the baselines largely saturates.

We appreciate the reviewer’s suggestion to examine this trade-off more closely. We will include these results in the revised manuscript and clarify how the confidence threshold allows practitioners to balance training cost and downstream accuracy.

Question 1: Use of a Frozen Model

We sincerely thank the reviewer for raising this important concern and apologize for the confusion. The actual PACS implementation used for all reported results does not rely on offline trajectories generated by a frozen model. Unfortunately, we inadvertently retained an outdated algorithm description in the manuscript (page 6 and page 22). We sincerely apologize for this oversight and have corrected the relevant descriptions to match the existing implementation (Now in the updated draft page 6 and page 22). Importantly, the implementation and all reported experimental results are NOT based on this incorrect version of the algorithm; this is a typo.

Actual implementation of PACS: At each selected state, the reference token remains the supervision target. When the model’s prediction disagrees with that reference, the loss provides corrective supervision at the state where the model is ready to commit. As training progresses, this supervision follows the model’s updated commitment decisions. The trajectory therefore determines where and when correction is applied; it is not itself a fixed generation path that the model is trained to reproduce. this is consistent with our uploaded code (example: Lines 664–686 of pacs-FB4E/training/campaigns/math…/pinned/trainer_clean_eotpad.py.).

Question 2: Potential inference speedup

We thank the reviewer for this helpful question. We additionally measure the average number of denoising forward passes (NFE) on LLaDA-1.5 under the same inference settings:

MethodGSM8K NFE ↓MATH NFE ↓HumanEval NFE ↓MBPP NFE ↓
Original LLaDA-1.569.84100.3175.4972.71
Semi-AR SFT65.3094.1667.6861.03
PACS*65.1293.9965.6760.53

PACS reaches a similar NFE reduction compared to semi-AR SFT. So both methods can improve inference speed a little bit, but PACS does not show significant gain.

Conceptually, PACS primarily focuses on improving learnability by directing supervision to where the model makes mistakes during decoding, rather than explicitly optimizing decoding speed. From our experience, accelerating decoding is more closely related to encouraging faster confidence convergence, for example, through certainty-forcing objectives such as the average predictive entropy minimization over correctly predicted tokens in dParallel (Chen et al., ICLR 2026).

Question 3: Training stage and initial model quality

We thank the reviewer for this thoughtful question. Although PACS is evaluated for supervised post-training, its motivation stems from a defining characteristic of DLMs. Unlike autoregressive models, where a gold reference determines an unambiguous conditioning prefix, DLMs' ability to predict distant masked positions introduces flexibility and ambiguity in the conditioning context, complicating learnability during training. Importantly, this parallel generation ability begins developing during earlier stages of training, rather than emerging only during post-training.

To further examine sensitivity to initialization, we provide an additional experiment on LLaDA 1.5, complementing the LLaDA-8B-Instruct and Dream-7B-Instruct results in the submission. The SFT methods use the same training data, and all methods follow the same evaluation protocol with a generation length of 256.

MethodGSM8KMATHHumanEvalMBPP
LLaDA-1.579.68%36.00%42.07%46.30%
Naive SFT79.83%36.80%45.73%49.03%
Semi-AR SFT79.98%37.20%47.56%51.75%
PACS80.36%38.60%51.22%54.86%

PACS improves over semi-AR SFT on all four benchmarks. Together with the results in the submission, this shows that PACS’s benefits are retained across the evaluated initial checkpoints and backbones.

Regarding iterative refresh, as clarified in Q1, our implementation constructs trajectories using the current model during training. Trajectory construction therefore already adapts to the evolving model, without requiring a separate refresh of a fixed offline dataset. We apologize for the incorrect frozen-model description and will correct it to match the implementation used for the reported results.

Question 4: Effect for block diffusion model

We thank the reviewer for raising this question. We have now evaluated PACS on SDAR-8B-chat-32b, a native block-diffusion model, and find that its gains extend to this architecture. We compare PACS with the original checkpoint, naive SFT, and semi-AR SFT across six benchmarks:

MethodGSM8KMATHHumanEvalMBPPLiveCodeBenchGPQA
Original SDAR86.13%45.00%58.54%70.82%41.93%29.80%
Naive SFT87.04%46.00%59.76%71.98%41.30%31.31%
Semi-AR SFT87.57%45.80%60.98%71.98%43.47%33.84%
PACS88.32%48.60%63.41%75.48%45.96%35.86%

PACS outperforms both SFT baselines on all six benchmarks, providing direct empirical evidence of its effectiveness on SDAR. These results extend our evaluation beyond the full-attention backbones studied in the submission and support PACS’s applicability to a native block-diffusion architecture. We will incorporate this experiment, together with the exact checkpoint and training and evaluation settings, into the revised paper.

Dear Reviewer skM9,

We are sincerely grateful for your thoughtful feedback, which has given us another opportunity to reflect on our work, a process that has been both challenging and rewarding. We truly appreciate your engagement and would be grateful if you could let us know whether our rebuttal has adequately addressed your concerns.

Reviewer_bgmaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines training–inference mismatch in SFT for masked diffusion language models along two axes of the training canvas — the reference tokens visible during supervision and the attention-visible canvas length — arguing that random masking yields supervision states unlike confidence-guided decoding and that canvas extent shifts the predictability of a fixed response. It proposes Predictability-Aware Canvas Selection and Supervision (PACS), which uses a frozen model to build reference-conditioned reveal trajectories, scores candidate canvas lengths by commitment-level reference cross-entropy, and fine-tunes on the lowest-loss canvas's pre-commitment states while keeping the response fixed. Across GSM8K, MATH500, MBPP, and HumanEval with LLaDA-8B-Instruct and Dream-7B-Instruct, PACS beats several SFT baselines at 128/256/512 generation tokens.

Critical strengths
  1. Clear diagnosis of training–inference mismatch in masked DLMs. Two mismatch sources are separated: the visible reference content at target-selection time and the attention-visible canvas extent. Controlled interventions (Fig. 3) show that withholding revealed context turns 37.3% of correct commitment predictions incorrect, while revealing the next reference token early corrects 51.2% of incorrect ones; Fig. 4 ties canvas extent to reference cross-entropy. Canvas construction therefore materially changes the prediction problem rather than being an inconsequential implementation choice.
  2. A coherent method, broadly evaluated, with component analysis. PACS selects canvas lengths by commitment-level reference cross-entropy and supervises the resulting pre-commitment states. Evaluation spans two backbones, four reasoning/code benchmarks, and three generation budgets; Table 3 separates canvas selection from confidence-guided reveal ordering, and Appendix B adds alternative-data and compute-matched comparisons—clarifying which components drive the gains.
Critical weaknesses
  1. The effect of trajectory-based supervision on generation diversity remains unclear. PACS selects a single canvas length from a limited candidate set for each training example and supervises the corresponding trajectory. Although this design aligns supervision with the model's confidence-guided decoding behavior, it remains unclear whether this will reduces output entropy or sampling diversity.
Questions for authors
  • The experiments could consider extending to block-wise DLMs, which are also one of the most practical forms of DLMs today, e.g., DiffusionGemma.
  • Would such training reduce sampling diversity and lower entropy? Because it sounds like, for a given specific sample, it seems to keep reinforcing certain trajectories, and if the number of constructed candidate trajectories is limited, is it prone to a noticeable entropy drop? (Consider reporting entropy and pass@k changes.)
  • How does PACS compared to trajectory-aware RL methods[1,2,3] for DLM? Given that PACS constructs multiple reveal trajectories and incurs substantial additional computation, could the authors clarify its connection to trajectory-aware RL for DLM, and whether similar entropy or diversity degradation occurs?

[1] Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models [2] Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models [3] d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation

7 comments

We sincerely thank reviewer for the thoughtful and constructive review. We appreciate the reviewer's concern and agree that accuracy improvements should be evaluated alongside the total post-training cost and pass-k accuracy. We are especially grateful that the reviewer recognized the contributions of our work.

To the best of our knowledge, our work is among the first to offer a unified perspective on DLM SFT, by jointly considering reference predictability, training-canvas construction (which determines learnability), and inference-time decoding behavior. Beyond the empirical improvements, we hope this conceptual perspective can inspire further research on more effective and better-aligned training methods for DLMs.

Below, we address each question and concern in detail. All the comments are very constructive and help strengthen our work. We also appreciate that none of the points raised indicate fundamental flaws in our core ideas or methodology.

Weakness: Effect on generation diversity

We thank the reviewer for raising this concern. PACS does not train on a fixed set of initial-model trajectories. Trajectories are constructed online using the current model’s confidence (we apologize for a typo in algorithm description), so the reveal order and pre-commitment states can change dynamically as the model is updated.

At each selected state, the reference token remains the supervision target. When the model’s prediction disagrees with that reference, the loss provides corrective supervision at the state where the model is ready to commit. As training progresses, this supervision follows the model’s updated commitment decisions. The trajectory therefore determines where and when correction is applied; it is not itself a fixed generation path that the model is trained to reproduce.

We complement the explanation with evaluations of multi-sample functional coverage and matched-state predictive entropy against standard SFT controls.

1. Multi-sample functional coverage (pass@k). We evaluate the original LLaDA-8B-Instruct model and the three fine-tuned checkpoints on HumanEval and MBPP. All methods use temperature 1, generation length 256, block length 32, and inference confidence threshold 0.9. We report the standard per-problem combinatorial pass@k estimator.

BenchmarkMethodpass@1pass@5pass@10
HumanEvalOriginal37.38%54.15%58.54%
HumanEvalNaive39.21%54.00%58.54%
HumanEvalSemi-AR42.68%56.86%60.37%
HumanEvalPACS45.49%58.57%62.80%
MBPPOriginal40.80%57.79%65.00%
MBPPNaive49.40%63.83%68.00%
MBPPSemi-AR49.60%63.41%69.00%
MBPPPACS52.70%66.22%72.00%

PACS’s gains persist at larger sampling budgets, indicating greater coverage of correct solutions.

2. Matched-state predictive entropy. To examine predictive concentration directly, we compare the fine-tuned models on the same 614 reference-conditioned block-entry states. Earlier reference blocks are visible and the current and future blocks remain masked. We compute full-vocabulary token entropy at temperature 1, average over valid reference positions within each problem, and weight the two benchmark means equally.

MethodMatched-state token entropy (nats)
Naive SFT3.2246
Semi-AR SFT2.9480
PACS3.3177

PACS has higher predictive entropy than both SFT controls on these shared states, providing evidence against greater predictive concentration under this evaluation. These results show higher multi-sample functional coverage and higher matched-state entropy for the evaluated PACS checkpoint. That means PACS do not achieve higher accuracy at the cost of sampling diversity.

Question 1: block diffusion models

We thank the reviewer for raising this question. We have now evaluated PACS on SDAR-8B-chat-32b, a native block-diffusion model, and find that its gains extend to this architecture. We compare PACS with the original checkpoint, naive SFT, and semi-AR SFT across six benchmarks:

MethodGSM8KMATHHumanEvalMBPPLiveCodeBenchGPQA
Original SDAR86.13%45.00%58.54%70.82%41.93%29.80%
Naive SFT87.04%46.00%59.76%71.98%41.30%31.31%
Semi-AR SFT87.57%45.80%60.98%71.98%43.47%33.84%
PACS88.32%48.60%63.41%75.48%45.96%35.86%

PACS outperforms both SFT baselines on all six benchmarks, providing direct empirical evidence of its effectiveness on SDAR. These results extend our evaluation beyond the full-attention backbones studied in the submission and support PACS’s applicability to a native block-diffusion architecture. We will incorporate this experiment, together with the exact checkpoint and training and evaluation settings, into the revised paper.

Question 2: Entropy and pass@k

The question has been answered in the weakness section.

Question 3: Relationship with trajectory-based RL

We thank the reviewer for highlighting this connection and appreciate the opportunity to further clarify the unique challenges of trajectory-aware SFT. We also had a brief discussion in Related Work: Trajectory-Aware Training and Reference Supervision (Lines 186–192).

Our trajectory-aware SFT shares with trajectory-aware RL the motivation to account for intermediate diffusion states. However, DLM SFT introduces two additional challenges that are largely absent in on-policy RL (e.g., GRPO).

1. Canvas-length mismatch. In DLM RL (e.g., The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models), rollouts and training use the same canvas length, avoiding unwanted supervision caused by differences between the model's generation style and a gold reference.

For example, with a 128-token canvas for a coding problem, the model is unlikely to generate a very concise code implementation in one rollout but a lengthy implementation followed by verbose explanations in another, as might occur with a 512-token canvas (Figs. 1A and 2).

In contrast, SFT may use a fixed 512-token canvas while the gold solution is much shorter (This is overlooked by existing SFT paper, where their choice of canvas length is either max canvas length of the training batch or all training samples, so typically a large fixed canvas length is common without PACS selection). This can produce an excessively high loss at the EOS position because the model's intended stopping point differs from the gold reference. Such a gradient might push the model to generate shorter responses on a 512-token canvas, rather than correcting errors in its predictions.

2. Trajectory-aware SFT needs to accommodate the teacher's gold reference. In RL, the model's decoding trajectory is naturally tied to its own generated solution. On the contrary, PACS uses model confidence only to **guide which positions to supervise ** and computes the supervision loss against the gold tokens (the context is still gold tokens; the decoding trajectory (confidence-based) only decides positions).

Additionally PACS’s improved pass@k supports its potential as a complementary SFT stage before RL. Compared with naive and semi-AR SFT, PACS achieves higher pass@k and higher matched-state predictive entropy. The pass@k gains indicate broader correct-solution coverage at the evaluated sampling budgets, providing evidence against the concern that its SFT gains diminish the useful solution coverage on which subsequent RL can build. These results make PACS a promising initialization for further RL training.

Trajectory-construction cost can be reduced greatly by lowering the training confidence threshold. Additional experiments at a threshold of 0.2 yield:

ConfigurationTraining ThresholdTraining TimeHumanEvalMBPP
Semi-AR block SFT (full run)—≈1 h 16 min43.90%48.60%
PACS (partial run)0.2≈1 h 05 min44.51%48.60%
PACS (full run)0.2≈3 h 15 min45.73%49.01%
PACS (full run)0.9≈14 h 55 min47.56%49.42%

At the lower threshold, the partial run matches semi-AR’s MBPP accuracy and slightly improves HumanEval accuracy with less training time. We will incorporate these results and clarify PACS’s relationship to trajectory-aware RL in the revised discussion.

Dear Reviewer bgma,

We are sincerely grateful for your thoughtful feedback, which has given us another opportunity to reflect on our work, a process that has been both challenging and rewarding. We truly appreciate your engagement and would be grateful if you could let us know whether our rebuttal has adequately addressed your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.