- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies inference-cost prediction for serving block-autoregressive diffusion language models. It introduces the Denoising Workload Surface (DWS), which represents execution probabilities over output blocks and within-block denoising steps. Combining these probabilities with deployment-specific profiled costs yields a request-cost estimate. The authors develop a prompt-only predictor using a factorized parameterization, coarse-to-fine training, and INT8 quantization for efficient CPU inference. Experiments evaluate cost-model fidelity, prediction accuracy, and downstream shortest-job-first scheduling on LLaDA2.0 models. The reported results show reduced prediction error and improved online latency and offline sample-completion performance, with additional evaluations of cross-dataset, cross-model, and cross-hardware transfer.
Critical strengths
- Structure-aware cost representation. The paper identifies why output length and total denoising steps are insufficient cost proxies for dLLMs. DWS captures the block–step execution structure and heterogeneous step costs, providing an interpretable basis for cost estimation.
- Demonstrated serving benefits. The lightweight CPU predictor improves both cost prediction and end-to-end scheduling performance in online and offline workloads. Transfer experiments further support its practical usefulness across deployment settings.
- Coherent design. The factorized predictor satisfies the structural constraints of the denoising process by construction, and its training loss is theoretically linked to the final cost-estimation error.
Critical weaknesses
- Unclear contribution of DWS. In Table 1, full DWS beats Direct Cost, but DWS without Stage A and the single-horizon variant do not. It is unclear whether Direct Cost receives the same Stage-A adaptation, so the gain cannot be attributed to the two-dimensional target versus multi-horizon training. Comparing both with the same Stage-A checkpoint and matched training budget would resolve this.
- Limited generality. All experiments use a single model family (LLaDA2.0), and the paper itself attributes the within-block cost variation that DWS relies on to MoE routing, noting it may be small for dense models (App. C.1, C.3).
Questions for authors
See Weakness
Rebuttal by Authors
Thank you for your constructive suggestions! We address your concerns below.
W1. Isolating DWS's advantage. DWS still outperforms Direct Cost under matched training conditions.
Following your suggestion, we initialize the Direct Cost predictor from the same Stage A checkpoint as DWS and use matched Stage-B training budgets.
After incorporating the same Stage A, Direct Cost improves from 432.29 to 408.75 ms in cost MAE, but still underperforms DWS (355.24 ms), suggesting that our design in Stage A benefits both predictors, while DWS retains a clear accuracy advantage. Moreover, unlike Direct Cost, DWS can transfer across hardware deployments without retraining the predictor.
W2. Generality Across Model Families and Architectures. We further show DWS's effectiveness beyond LLaDA2.0 and clarify how model architecture affects its benefits.
W2 (a). Cross-family generality.
Overall, these results suggest the generality of DWS across different dLLM model families, spanning cost prediction and serving.
W2 (b). Effect of model architecture.
We will incorporate the above results and analyses in the revision. Thank you for your time and thoughtful feedback!