- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Spatio-Temporal Pruning (STP), a framework intended to make reinforcement learning for masked diffusion language models cheaper and more stable. STP prunes along two axes: spatial pruning fixes a fraction of response tokens to anchor values from earlier generations, and temporal pruning skips the final denoising steps and fills all remaining masks in one parallel forward pass. The paper establishes a lower-variance upper-bound for the ELBO estimator, a bound on the output change from early completion, and bounds linking ELBO-difference error to the GRPO objective.
Critical strengths
The proposed method exploits dLLM structure in a simple and modular way. Both pruning operations use properties unique to masked diffusion models, such as conditioning on arbitrary fixed tokens, and filling all remaining masks in a single forward pass. The authors also provide theoretical analysis on the variance reduction and reduced sampling cost.
Critical weaknesses
The efficiency claims in the experiments are fragile. On GSM8K and Countdown, STP’s training-time advantage over Diffu-GRPO is only 1.4% and 3.4% (Table 1). Those figures also exclude the one-time cost of building the static traces. When it's included, the paper’s own end-to-end numbers are 150,500s on GSM8K and 148,299s on Countdown (line 405), slower than Diffu-GRPO’s 146,641s and 147,100s (Table 1). Also, the accuracy gains on MATH and GSM8K appear marginal, raising questions about whether the reduction in training time achieved by STP justifies the potential loss in accuracy.
Questions for authors
- Please use \citep{} and \citet{} correctly.
- The title of Section 2.1 should be "GRPO for LLMs", not "GRPO for dLLMs".
- On line 182, how is chosen? Is ? Is ?
- Diffu-GRPO achieves the same accuracy as the base model on MATH in Table 1, which seems unexpected given the report in the original d1 paper.
- The efficiency gains appear marginal, and accuracy may be a more important consideration during training. The longer training time may be an acceptable trade-off if it leads to better accuracy.
- The main experiments are conducted on LLaDA-8B-Instruct. Will the authors consider evaluating their method on more advanced DLMs, such as Efficient-DLM?
- Surprised to see SPG achieves 0 accuracy in Table 1. (SPG is known to perform well on Countdown. Please double check.
4 comments
Response to Reviewer QDMA (Part 2 / 3)
Q5. On line 182, how is chosen? Is ? Is ?
A5: We consider two ways of choosing : random selection and confidence-based selection. The main experiments adopt Low Confidence Retention, which retains low-confidence tokens as active positions and fixes high-confidence anchor tokens; Random Selection instead randomly selects fixed tokens among candidate positions, with results reported in Table 7.
denotes the set of fixed token position indices, and denotes the complete anchor sequence. Hence it is not the case that ; for , position uses the anchor token , while positions with continue to participate in denoising and exploration.
denotes the target spatial pruning ratio, and denotes the actual fraction of fixed positions. Under Random Selection, ; under Low Confidence Retention, we select approximately high-confidence positions to fix, so the actual ratio is affected by the sequence length and rounding. Consequently, and are typically close but not necessarily exactly equal. In the paper, uses the actual ratio , i.e., .
Q6. Diffu-GRPO achieves the same accuracy as the base model on MATH in Table 1, which seems unexpected given the report in the original d1 paper.
A6: The fact that Diffu-GRPO matches the base model on MATH is not contradictory to the original paper. The results in the original paper that exceed the base model come from selecting the checkpoint with the highest test accuracy every 100 steps, whereas we report the performance of the final checkpoint, a convention consistent with [1]. Figure 7 of the d1 paper shows that the reward trajectory of LLaDA+Diffu-GRPO during MATH training varies considerably and is below or on par with the initial value for most of the time, indicating that the training process is subject to fluctuations.
Q7. The efficiency gains appear marginal, and accuracy may be a more important consideration during training. The longer training time may be an acceptable trade-off if it leads to better accuracy.
A7: We agree with the reviewer that training efficiency should not come at the expense of accuracy or training stability. The design goal of STP is precisely to simultaneously reduce the computational overhead of rollout and the noise in ELBO estimation, thereby improving optimization while increasing training efficiency.
Theoretically, Theorem 4.1 characterizes the reduction in sampling computation brought by temporal pruning; Theorem 4.2 shows that spatial pruning reduces the variance upper bound of the ELBO estimate to times the original; Eqs. (13)–(15) analyze the resulting bias–variance trade-off; and Theorem 4.5 further characterizes the effect of the error and variance of the ELBO difference on the GRPO surrogate. These analyses show that STP can improve training stability by reducing estimation noise, rather than merely reducing computation.
Empirically, STP achieves accuracy no lower than the baselines on all three benchmarks (relative to Diffu-GRPO: +3.8, +0.8, and +40.6 percentage points on MATH, GSM8K, and Countdown, respectively), while reducing the variance of the paired current–old ELBO by 79.1% (Table 2) and requiring shorter training time. Therefore, the efficiency gains of STP are not obtained at the expense of optimization quality: STP achieves more stable training with lower variance, thereby reaching higher accuracy in less time.
Response to Reviewer QDMA (Part 3 / 3)
Q8. The main experiments are conducted on LLaDA-8B-Instruct. Will the authors consider evaluating their method on more advanced DLMs, such as Efficient-DLM?
A8: We thank the reviewer for this suggestion. To examine the applicability of STP across different DLM architectures, we further conducted experiments on Efficient-DLM-4B on GSM8K and Countdown. To ensure consistency in comparisons across methods, all settings use the same task prompts, training configurations, and full-sequence sampler. We found that the official Efficient-DLM sampler suffers from severe hallucination on these two datasets, with base accuracy close to 0, making RL training infeasible; we therefore uniformly switched to the full-sequence sampler for decoding.
Table1: Accuracy and online RL training time on Efficient-DLM-4B.
| Methods | GSM8K Accuracy | GSM8K Time(s) | Countdown Accuracy | Countdown Time(s) |
|---|---|---|---|---|
| Efficient-DLM-4B | 49.6588% | - | 4.2969% | - |
| GRPO w/ ELBO | 35.9363% | 54358 | 1.5625% | 63135 |
| SPG | 1.5163% | 47482 | 0.0000% | 54275 |
| Diffu-GRPO | 28.7339% | 48777 | 4.6875% | 55509 |
| GRPO w/ ELBO+STP(Ours) | 49.9621% | 46108 | 6.2500% | 53012 |
On GSM8K, the base accuracy of Efficient-DLM-4B is 49.66%, which increases to 49.96% with STP; the other three RL methods all exhibit clear degradation relative to the base model. We conjecture that this is because the training signal of this base model on GSM8K is relatively sparse, so methods with larger variance are more affected by noise under limited signal, whereas STP maintains training stability by reducing estimation variance. The online RL training time of STP is 46,108 s; relative to GRPO with ELBO and Diffu-GRPO, STP reduces training time by approximately 15.2% and 5.5%, respectively.
On Countdown, STP improves accuracy from 4.30% for the base model to 6.25%, with a training time of 53,012 s; relative to GRPO with ELBO and Diffu-GRPO, STP reduces training time by approximately 16.0% and 4.5%, respectively.
These results show that STP can likewise improve both training efficiency and final accuracy on Efficient-DLM-4B. This provides additional evidence for the transferability and training stability of STP across different DLM architectures.
Q9. Surprised to see SPG achieves 0 accuracy in Table 1. (SPG is known to perform well on Countdown. Please double check.
A9:We thank the reviewer for this reminder. We have re-checked the implementation, training logs, and evaluation results of SPG. Under the unified experimental setting of this paper, SPG uses the token-level variant, and we report the final checkpoint at the end of training; all three independent runs on Countdown exhibited training collapse, resulting in a final accuracy of 0.0%. SPG removes the KL divergence term and performs μ gradient updates on the same batch of samples while dropping the importance sampling correction, making training more prone to model collapse. This instability is already reflected in the original SPG paper: its Figure 9 shows a sudden increase in generation length during training on Countdown, and issues in the official repository also report training collapse. Since we report the final checkpoint, if collapse occurs in the middle or later stages of training, its state will be directly reflected as 0.0% accuracy on Countdown and below-base performance on GSM8K.
[1] DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation. ICLR 2026
We hope the above clarifies the scope of our efficiency claims—STP's advantage is established relative to GRPO with ELBO and, more importantly, it is orthogonal and stackable on top of Diffu-GRPO (Appendix Table 6), where it reduces end-to-end training time by 25.1% with a 1.6-point accuracy gain—and that our re-examination of the SPG and Diffu-GRPO results, together with the additional Efficient-DLM-4B experiments, addresses your concerns about the plausibility and generality of the reported numbers. We would be grateful if you would reconsider the rating should these clarifications and additional results address your concerns. Thank you once again for your careful and constructive review.
Dear Reviewer,
Thank you for your careful review. We have submitted point-by-point responses to your questions and, following your suggestion, added experiments on Efficient-DLM-4B. We would like to invite you to review them at your convenience and let us know whether these clarifications and new results address your main concerns. Following the numbering of your original questions, the key responses are as follows:
-
On the trade-off between efficiency and accuracy (main concern and Q5). Relative to GRPO w/ ELBO, STP improves accuracy by 2.4, 0.5, and 29.7 percentage points on MATH, GSM8K, and Countdown, respectively; after accounting for anchor costs, total training time is still reduced by 20.0%, 7.7%, and 9.9%, respectively. In these experiments, time savings are accompanied by accuracy improvements. We also explicitly acknowledge that, after accounting for anchors, GRPO w/ ELBO + STP does not outperform Diffu-GRPO in single end-to-end time on GSM8K and Countdown. Furthermore, Table 6 shows that, on MATH, adding STP to Diffu-GRPO still reduces end-to-end time by 25.1% relative to Diffu-GRPO even after accounting for anchor construction costs, while improving accuracy by 1.6 percentage points.
-
On validation with a more advanced DLM (Q6). We have added experiments on Efficient-DLM-4B. Under the setting where all methods uniformly use the full-sequence sampler, STP achieves 49.96% and 6.25% on GSM8K and Countdown, respectively, exceeding other algorithms, and attains the lowest online training time among the compared RL methods. This directly addresses your suggestion regarding validation on extended models and provides additional evidence for cross-model applicability.
-
On the results of Diffu-GRPO and SPG (Q4, Q7). We have clarified that this paper adopts the final-checkpoint evaluation protocol and discussed the difference between this and results reported with the best checkpoint. In addition, we re-examined the implementation, training logs, and evaluation of SPG: under the token-level setting of this paper, all three independent runs on Countdown exhibited training collapse, with a final accuracy of 0.0%.
-
On the choice of fixed positions and notation definitions (Q3). The main experiments fix high-confidence anchor positions while retaining low-confidence positions for continued exploration. We have distinguished the fixed position index set from the full anchor sequence , as well as the target pruning ratio from the actual ratio . The target and actual ratios need not be strictly equal; initialization uses the actual ratio, i.e., . Results for the random-selection control are shown in Table 7.
-
On citation format and section titles (Q1, Q2). We accept your suggestions and have corrected the use of
\citep{}and\citet{}in the revision, and corrected the title of Section 2.1 to "GRPO for LLMs".
We hope that these clarifications, especially the Efficient-DLM experiments added at your suggestion and the comparisons that include anchor costs, will help you reassess the empirical contributions of this paper. If key questions remain, please point them out and we will be glad to respond further; if your main concerns have been alleviated, we hope you will reconsider your evaluation and score in light of the above evidence. Thank you again for your time, and we look forward to your feedback.
Response to Reviewer QDMA (Part 1 / 3)
We really appreciate the reviewer for the detailed and constructive feedback. We are glad that you recognized the way our method exploits dLLM structure in a simple and modular manner, and that you found the theoretical analysis on variance reduction and sampling cost useful. We also take seriously your concerns regarding the robustness of our efficiency claims and the plausibility of several reported numbers. We believe our response has addressed most of your concerns and we respectfully ask if you're comfortable to re-consider the rating in light of our efforts and dedication. Below, we provide detailed responses to all the comments:
Q1. On GSM8K and Countdown, STP’s training-time advantage over Diffu-GRPO is only 1.4% and 3.4% (Table 1). Those figures also exclude the one-time cost of building the static traces. When it's included, the paper’s own end-to-end numbers are 150,500s on GSM8K and 148,299s on Countdown (line 405), slower than Diffu-GRPO’s 146,641s and 147,100s (Table 1).
A1: We sincerely appreciate your valuable comments. We agree that, in a single end-to-end comparison against the fastest baseline, Diffu-GRPO, STP does not have an advantage in end-to-end training time on GSM8K and Countdown once the one-time anchor construction cost is included.
Relative to GRPO with ELBO, after accounting for the anchor construction cost, STP reduces end-to-end training time by 20.0%, 7.7%, and 9.9% on MATH, GSM8K, and Countdown, respectively. Meanwhile, STP achieves accuracies of 36.6%, 81.0%, and 66.0%, respectively, all higher than those of Diffu-GRPO (32.8%, 80.2%, and 25.4%), with the improvement on Countdown being particularly pronounced.
Anchors are task-level static resources shared by all prompts and training runs of the same task, so their construction cost can be amortized across multiple experiments on the same task. In addition, STP is orthogonal to the underlying RL objective and can be stacked on top of Diffu-GRPO. Table 6 verifies this on MATH: with STP added, the online training time of Diffu-GRPO is reduced by 29.8% (excluding anchors) and accuracy improves by 1.6 percentage points; after accounting for the anchor construction cost, the end-to-end training time is still reduced by 25.1% (including anchors).
Q2. Also, the accuracy gains on MATH and GSM8K appear marginal, raising questions about whether the reduction in training time achieved by STP justifies the potential loss in accuracy.
A2: We thank the reviewer for this valuable suggestion. The reduction in training time achieved by STP does not come at the cost of accuracy: STP attains accuracies of 36.6%, 81.0%, and 66.0% on MATH, GSM8K, and Countdown, respectively, improving over GRPO with ELBO by 2.4, 0.5, and 29.7 percentage points. Relative to GRPO with ELBO, after accounting for the anchor construction cost, STP reduces end-to-end training time by 20.0%, 7.7%, and 9.9% on the three tasks, respectively. This efficiency gain arises together with the accuracy gain, consistent with the reduced variance of the ELBO-difference estimator. Figure 2 shows the distribution of the ELBO estimation variance during training, where several high-variance outliers can be observed; high-variance estimates may produce extreme ratios that fall into the clipping region, thereby wasting effective learning signals. By reducing the estimation variance and the frequency of high-variance outliers, STP makes the ratio and the subsequent gradient updates more stable. Table 2 shows that STP reduces the variance of the paired current–old ELBO by 79.1%. These results are consistent with the observation that STP maintains and improves final accuracy while reducing training time.
Q3 and Q4. Please use \citep{} and \citet{} correctly. The title of Section 2.1 should be "GRPO for LLMs", not "GRPO for dLLMs".
A3 and A4: Thanks for the catch. We will correct these typos and conduct a complete check in the revision.