- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper found that there are conflicts among the individually confident predictions during parallel decoding in dLLMs. To handle this problem, this paper proposed RAPD, which is a training-free method to deal with selections based on energy. The energy combines each position's unary commitment evidence with signed pairwise interactions. The results on LLaDA-8B-Instruct and Dream-7B-Instruct across 4 benchmarks show that RAPD achieves the fewest forward passes and the highest throughput on most of the experiments.
Critical strengths
- Well-motivated. The collision example shows why independent per-position thresholding fails. Hence, it is natural to frame commit-set selection as an energy with attractive and repulsive interactions.
- Light and practical. RAPD is a training-free method and needs no extra forward passes.
- Strong results in efficiency. On LLaDA-8B, RAPD has the lowest NFE and highest TPS on all four benchmarks with a single configuration.
Critical weaknesses
- Limited generalization. The paper noted that their method's weights were selected on HumanEval, where it also achieved the best performance. However, on other benchmarks, the accuracy is at or near the bottom among the compared methods, which is not stable.
- Mismatch between the motivation and the method's effect. The paper motivates RAPD to reduce prediction collisions. Repulsion is the only mechanism in RAPD that could prevent a conflicting commitment, but repulsion alone does not improve on no coupling.
- Limited robustness. All main results use a single generation shape (length 256, block size 32). However, when the generation length is increased to 512 on HumanEval, every baseline improves by 4.9-6.1 points, whereas RAPD shows no improvement.
Questions for authors
- Why were the weights of RAPD tuned on HumanEval? If the weights were selected on another dataset, would the resulting configuration and the reported gains remain the same?
- Why does RAPD have no benefit from a longer generation length on HumanEval? Does re-tuning the threshold or weights for longer lengths recover this gap?
- Is RAPD compatible with KV-Cache (e.g., Fast-dLLM’s prefix or dual cache)? How does performance compare in that setting and at larger batch sizes?
3 comments
[W3/Q2] Performance at longer generation lengths.
We sincerely thank the reviewer for highlighting this issue. The reviewer's feedback helped us close the accuracy gap at length 512 by revisiting the decoding settings.
We'd like to note that increasing the generation length also substantially increases the decoding cost. For the three baselines, extending the length from 256 to 512 improves HumanEval accuracy by 4.9–6.1 percentage points but requires 2.03–2.11× as many forward passes. The longer setting therefore represents a substantially larger computational budget, and its accuracy gains should be assessed alongside this cost. Given our focus on efficient decoding, we investigated whether RAPD can recover accuracy at this longer length while retaining its forward-pass savings.
The accuracy gap at length 512 is closed by reselecting the commitment threshold and per-block step cap on development data. Here, we take the MBPP development set at length 512 and the same selection rule as in Q1. The resulting HumanEval score increases significantly from 66 to 77 of 164 problems at generation length 512.
| Method | HumanEval (164) | GSM8K (1,319) |
|---|---|---|
| RAPD (paper settings) | 66 / 117.2 | 1,027 / 70.0 |
| RAPD (dev-selected threshold and cap) | 77 / 139 | 1,025 / 81 |
| Fast-dLLM | 74 / 164.9 | 1,031 / 95.1 |
| LocalLeap | 75 / 124.3 | 1,026 / 72.8 |
| DAWN | 72 / 130.0 | 1,009 / 70.4 |
The new RAPD setting shows no significant accuracy difference from any baseline on either benchmark, while requiring approximately 15–16% fewer forward passes than Fast-dLLM.
Our analysis indicates that the original decoding settings commit too aggressively at length 512. The commitment threshold and per-block step cap should be selected considering the generation length and block size; the threshold controls how aggressively positions are committed, while the step cap sets the decoding budget for each block. We will clarify this distinction in the revised manuscript.
[Q3] Compatibility with KV caching and larger batch sizes.
KV caching. Yes, RAPD is compatible with Fast-dLLM’s PrefixCache and DualCache. We evaluated both decoding methods using the same cache implementation on LLaDA-8B-Instruct with a single RTX 4090.
Each entry reports RAPD / Fast-dLLM. Accuracy is the number of problems solved, and TPS is measured end to end at batch size 1.
| Benchmark | Cache | Problems solved | NFE | TPS |
|---|---|---|---|---|
| HumanEval (164) | Prefix | 70 / 69 | 70.5 / 84.5 | 88.9 / 76.0 |
| HumanEval (164) | Dual | 58 / 58 | 77.9 / 90.7 | 61.5 / 55.1 |
| MBPP (500) | Prefix | 203 / 211 | 59.9 / 69.6 | 56.9 / 50.4 |
| MBPP (500) | Dual | 189 / 193 | 59.3 / 69.3 | 45.0 / 40.3 |
| GSM8K (1,319) | Prefix | 1,034 / 1,025 | 68.2 / 81.2 | 63.9 / 54.7 |
| GSM8K (1,319) | Dual | 1,035 / 1,042 | 72.2 / 85.3 | 52.5 / 46.1 |
| MATH (5,000) | Prefix | 1,615 / 1,649 | 86.6 / 100.8 | 65.4 / 57.4 |
| MATH (5,000) | Dual | 1,615 / 1,635 | 92.8 / 107.1 | 50.3 / 45.0 |
Across all eight combinations, RAPD requires 13–17% fewer forward passes and achieves 12–17% higher TPS, with no significant accuracy difference from Fast-dLLM. These results demonstrate the effectiveness of RAPD on top of KV-cache acceleration.
Batch size. We conducted a trace-based analysis of block-synchronous batching using per-problem decoding traces collected under PrefixCache. We formed random batches and counted the maximum number of steps required by their constituent sequences within each block.
At the paper’s settings, the estimated NFE reduction over Fast-dLLM increases from approximately 30% at batch size 1 to 35% at batch size 16, using all 164 HumanEval problems and the GSM8K problems. RAPD’s per-block step cap limits the waiting time caused by slower sequences.
We sincerely thank the reviewer again for suggesting these valuable experiments, which helped us strengthen the evaluation of RAPD under practical inference settings.
Dear Reviewer WLcs,
Thank you again for your time and constructive feedback.
Your suggestions have substantially improved our paper. In response, we made several additions, most notably repeating the configuration selection on held-out development data, clarifying the role of repulsion through controlled ablations, re-selecting the decoding settings for longer generations, and evaluating RAPD with KV caching. The revised manuscript has been uploaded, with all changes highlighted in red.
As the discussion period ends soon, we would sincerely appreciate your feedback on whether our responses have addressed your concerns. We would be glad to answer any remaining questions.
Best regards,
The Authors
We thank the reviewer for the thoughtful review and helpful comments.
[Q1/W1] Configuration selection on a different dataset.Thank you for the question. HumanEval served as our initial development benchmark for selecting the configurations, which we then fixed across all four tasks. To directly test whether the gains depend on this choice, we repeated configuration selection using development data, without using test results for selection.
Using only the MBPP training and validation splits (464 problems), we reselected the coupling parameters, unary criterion, readout layer, kernel width, and per-block step cap. We selected the setting with the highest development accuracy, breaking ties by fewer forward passes. Most settings remained unchanged: κ increased from 2 to 4, with the repulsion weight adjusted to preserve the coupling budget, and the per-block step cap increased from 16 to 32.
We evaluated this single configuration across all four test sets; each entry below reports (problems solved / NFE).
The development-selected configuration uses 16–30% fewer forward passes than Fast-dLLM, with no significant accuracy difference on any benchmark. These results show that RAPD’s efficiency gains persist when configuration selection uses separate development data.
[W2] Effect of Repulsion.Thank you for this observation. We apologize for any confusion. The role of repulsion is to preserve accuracy as attraction expands the commit set. This is the effect stated in the abstract: repulsion "mitigates the accuracy loss from attraction-only decoding."
The limited standalone benefit is consistent with the stability of high-confidence predictions under threshold decoding. We first examined co-committed pairs on HumanEval and 200 problems each from MBPP and GSM8K. Committing one position and rerunning the model changed the other position’s most likely token in only 370 of 472,453 pairs (0.08%).
The attraction mechanism expands the commit set to include positions below the confidence threshold. Repulsion becomes effective in this more aggressive regime, as shown by the controlled ablation below. Each entry reports (problems solved / NFE).
Adding repulsion to attraction improves accuracy on both code benchmarks, while retaining approximately 13% fewer forward passes than no coupling. On HumanEval, it also reduces adjacent same-token co-commitments from 822 to 536, with the same-token share among all co-committed pairs decreasing from 5.35% to 4.25%.
These results support the intended interaction: attraction increases parallel commitment, while repulsion recovers the accuracy lost under attraction-only decoding and preserves the efficiency gain. We will clarify these complementary roles in the revision.