W1. The comparison with existing adaptive decoding methods is insufficient. In recent years, several works have studied adaptive unmasking for dLLMs, such as information-gain-based sampling [1], learned unmasking policies [2], Frontier Gating [3], and Entropy-Bounded Unmasking [4]. These works are directly related to the research objective of PACE, but this paper mainly compares with Confidence Order, Tempered Selection, and fixed AR Order. Although different methods may have different training and inference overheads, the authors are still advised to add several representative strong baselines under unified models, sampling settings, and computational budgets. In addition, Top-k/Top-p position sampling and the Adaptive Position Temperature discussed in the theoretical part of the paper are also worth direct comparison. These experiments would help more fully demonstrate the advantages of PACE over existing adaptive decoding strategies.
W2. The connection between the core algorithm and existing Top-H Decoding deserves further clarification. The main operation of PACE is to sort position probabilities and then select the largest prefix set satisfying H(qS)≤αH(q), followed by renormalized sampling within the retained set. This is very close to the core mechanism of Top-H Decoding [5], which performs truncation based on a relative-entropy budget; the main difference is that the truncation target is shifted from the token probability distribution to the unmasking position distribution. Although this transfer may be applicable to the decoding-order problem specific to dLLMs, the authors are advised to further explain what new technical challenges position selection introduces compared with ordinary token sampling, and what the independent methodological contribution of PACE is relative to existing Top-H mechanisms.
W3. The current experiments still have limitations in evaluating practical inference efficiency and generation diversity. The main experiments adopt a decoding setting in which only one token is determined per step, so generating a sequence of length 256 requires 256 model forward passes. The appendix adopts a confidence-thresholding method that dynamically commits multiple tokens and already achieves comparable accuracy at a notably lower NFE; therefore, the accuracy–compute advantage of PACE still needs further verification. In addition, in the supplementary experiments with Ttok=0.6, Tempered Selection outperforms PACE under larger sampling budgets. For example, on LLaDA/MBPP, their Pass@64 values are 73.3% and 60.0%, respectively. This suggests that PACE may focus more on improving single-sample accuracy and may have some impact on sampling coverage. The authors are advised to further discuss this trade-off and to supplement parallel decoding and more complete Pass@k–compute comparisons.
References
[1] Yang, K., et al. (2026). Improving Sampling for Masked Diffusion Models via Information Gain. ICML 2026.
[2] Jazbec, M., et al. (2026). Learning Unmasking Policies for Diffusion Language Models. ICML 2026.
[3] Yeom, J., et al. (2026). Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models. arXiv:2608.05687.
[4] Ben-Hamu, H., et al. (2025). Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking. NeurIPS 2025.
[5] Baghaei Potraghloo, E., Azizi, S., Kundu, S., & Pedram, M. (2025). Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation. NeurIPS 2025.