- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the training-inference mismatch in SFT for Diffusion Language Models (DLMs), where random masking and improper canvas extent can misdirect supervision. To resolve this, the authors propose PACS, a predictability-aware method that selects the optimal canvas length and supervises reference tokens immediately prior to confidence-guided revelation. Extensive experiments demonstrate that PACS outperforms standard SFT baselines across several reasoning and code generation benchmarks.
Critical strengths
The core idea is insightful and well-motivated, bringing a novel decoding-aware perspective to canvas construction during DLM training. Furthermore, the proposed method achieves noticeable performance improvements across multiple benchmarks.
Critical weaknesses
- Limited Experimental Scale and Generalizability. The current empirical evaluation is relatively light. The experiments should be expanded to include additional datasets and a broader range of modern DLM architectures/backbones (e.g., LLaDA 1.5, LLaDA 2.x, or larger-scale models). Without these, it is difficult to assess whether the proposed method can effectively scale or hold up across diverse base models.
- High Search Overhead and Scaling Bottleneck in Long-Context and Large-Scale Scenarios. The method relies on predictability-aware candidate search to select the optimal canvas. However, in the context of long-context scenarios and large-scale training data, the computational and token-level overhead for constructing, searching, and evaluating multiple candidate windows will explode. This prohibitive search and preprocessing overhead makes it extremely difficult for the approach to scale effectively to industrial-grade datasets and long sequences.
- New Latent Training-Inference Mismatch via Flawed Model Priors. The fundamental assumption behind PACS can still introduce latent training-inference mismatch. Prior to SFT, the model's pre-existing distribution dictates its prediction confidence, which may diverge from the ground truth. For instance, the model might assign a high confidence () to token at a position where the gold label is . Relying on the model's confidence to select canvases and trajectories can be flawed, as it trusts the model's "confidently wrong" belief to shape the supervision context.
Questions for authors
Same as "Critical Weaknesses"
4 comments
Weakness 2: High search overhead and scaling bottleneck
We thank the reviewer for raising this important practical concern. While the main paper focuses on clearly presenting our novel conceptual contribution of illustrating how standard SFT can misdirect supervision in DLMs, we recognize the importance of computational efficiency in practice. PACS adds a bounded, adjustable computational multiplier over baseline methods, preserving its asymptotic scaling with sequence length. Below, we provide additional results demonstrating substantially improved efficiency, with accuracy that matches or exceeds best baseline at comparable training cost. Additional computation is an optional user choice for pursuing further accuracy gains, allowing practitioners to select a configuration that fits their computational budget.
- To the best of our knowledge, PACS's canvas-length search addresses a major source of training–inference mismatch overlooked by all existing SFT methods for DLMs; therefore, we further highlight its novel and significant conceptual contribution.
We view canvas-length search as a flexible user option rather than a fundamental operational bottleneck. Each additional level of refinement in the canvas-length search yields further improvements in training performance (Fig. 6), demonstrating the value of more precise canvas selection while allowing users to balance their computational resources. For example, users with limited training time can choose a coarser search granularity.
In contrast, the commonly used batch-max or dataset-max canvas lengths show clear disadvantages compared with predictability-aware canvas selection (Figs. 5 and B.1). Importantly, these benefits are not limited to PACS: simple canvas-length selection also improves baseline methods (Fig. B.1), highlighting the broader significance of this overlooked issue.
- Rather than viewing PACS as unnecessarily expensive or inherently difficult to scale, we hope to emphasize that its higher computational cost per training pass does not necessarily translate into higher overall training cost. Instead, its computation is directed toward addressing a broader, fundamental challenge: the training–inference mismatch in DLMs, which makes effective supervision difficult.
While other SFT methods (e.g., vanilla SFT) are computationally cheaper per training pass, they extract less effective learning signal from the same data and tend to reach a performance ceiling even with prolonged training. Therefore, we emphasize comparisons on the same dataset under matched training time (Appendix Table B.4). PACS continuesly improves with further training, whereas the baselines largely saturate, pushing the achievable accuracy frontier higher.
- To address the reviewer's valuable practical concern regarding training cost, we further offer an additional option to substantially reduce training cost of PACS while retaining competitive accuracy. We view the training confidence threshold as a flexible user choice for balancing computational cost and performance. This configuration substantially reduces training cost while maintaining competitive performance, as shown below:
| Configuration | Training Threshold | Training Time | HumanEval | MBPP |
|---|---|---|---|---|
| Semi-AR block SFT | — | ≈1 h 16 min | 43.90% | 48.60% |
| PACS (partial run) | 0.2 | ≈1 h 05 min | 44.51% | 48.60% |
| PACS (full run) | 0.2 | ≈3 h 15 min | 45.73% | 49.01% |
| PACS (full run) | 0.9 | ≈14 h 55 min | 47.56% | 49.42% |
The partial run matches or exceeds best baseline on both benchmarks with less training time. Further training improves both scores, while the full run at threshold 0.2 remains approximately 4.6× faster than the original configuration. These results show that PACS can achieve competitive accuracy at a training cost comparable to semi-AR, and benefit from additional training in the evaluated setting while the baselines largely saturates.
Weakness 3: Training inference mismatch via flawed model prior
We sincerely thank the reviewer for raising this concern, which gives us an opportunity to clarify an important distinction in PACS. PACS's trajectory-guided supervision is designed to correct errors arising from the model's existing predictions, rather than reinforce potentially flawed model priors. In particular, model confidence guides where to supervise, not which predictions to trust.
In the reviewer’s example, confidently wrong predictions are precisely a case PACS is designed to correct. When the model confidently predicts A while the reference is B, PACS applies the loss at the pre-commitment state, with B as the gold-label target. The low probability assigned to B produces a high reference loss precisely where the model would make the mistake of predicting A. The incorrect prediction A is never accepted as the training prefix.
However standard SFT can misdirect supervision because its masking pattern need not follow the model's decoding trajectory(Section 4.1, Figure 3).
We sincerely appreciate the reviewer's thoughtful feedback, which touches on the central observations and contributions of our work, and would be happy to provide further clarification or discuss any remaining questions.
Dear Reviewer dp5z,
We are sincerely grateful for your thoughtful feedback, which has given us another opportunity to reflect on our work, a process that has been both challenging and rewarding. We truly appreciate your engagement and would be grateful if you could let us know whether our rebuttal has adequately addressed your concerns.
We sincerely thank reviewer for the thoughtful and constructive review. We appreciate the reviewer's concerns and the opportunity to further clarify our contributions.
To the best of our knowledge, our work is among the first to offer a unified perspective on DLM SFT by jointly considering reference predictability, training-canvas construction (which shapes learnability), and inference-time decoding behavior. Beyond the empirical improvements, we hope this conceptual perspective can inspire further research on more effective and better-aligned training methods for DLMs.
Below, we address each question and concern in detail. All the comments help us further clarify our work, and the reviewer's questions highlight some of the key distinctions of our method. We also appreciate that none of the points raised indicate fundamental flaws in our core ideas or methodology.
Weakness 1: Experimental scale and generalizability
We thank the reviewer for highlighting the need for broader evaluation. We have expanded our experiments to two additional models, LLaDA-1.5 and SDAR-8B-chat-32b, and two additional benchmarks, LiveCodeBench and GPQA. These experiments complement the LLaDA-8B-Instruct and Dream-7B-Instruct results in the submission. LLaDA-1.5 tests whether the gains persist across checkpoints within the LLaDA family, while SDAR extends the evaluation to a block-diffusion architecture. The additional benchmarks broaden coverage to contest programming and scientific reasoning.
LLaDA-1.5
SDAR
PACS outperforms both naive and semi-AR SFT on all six benchmarks for both additional models. These consistent gains strengthen the evidence that its benefits extend across initial checkpoints, model architectures, and evaluation tasks, including both newly added benchmarks. We will incorporate these experiments and their evaluation protocols into the revised paper.