- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TIPS uses a roughly 629K-parameter, two-layer DiT controller to manage token visibility in a frozen LLaDA model. The policy can reveal tokens and remask unreliable ones; a logit shift aligns its expected visible fraction with a prescribed progress schedule. It is trained with supervised warmup followed by GRPO using terminal correctness rewards. On 1,819 GSM8K and MATH-500 questions, TIPSfull reaches 58.46% accuracy at 32 decoder calls, compared with 46.34% for WINO. The main results use three training seeds and three inference draws per seed. The paper also studies remasking, policy architecture, and transfer across models and block sizes.
Critical strengths
- The paper separates visibility control from the frozen model’s token prediction and allows already revealed tokens to be withdrawn. The policy is small, and its training procedure and visibility schedule are well specified.
- TIPS improves accuracy over several baselines at low and moderate decoding budgets. The main results use three training seeds and include ablations on remasking and supervised warmup.
Critical weaknesses
- The appendix states that Fast-dLLM’s threshold is tuned using test-set rollouts to reach a target mean NFE, whereas TIPS’s is selected using validation data. Although no answer labels are used, the baseline uses the test-prompt distribution for budget matching, making the protocols asymmetric. Fast-dLLM’s 4-call result is also imputed from its 8-call result.
- Evaluation is concentrated on one LLaDA backbone, mathematical tasks, and 256-token responses. Decoder-call counts do not directly measure wall-clock time; the reported policy time is about 8 ms but excludes the decoder and feature extraction, so the end-to-end inference benefit remains unclear.
- The paper does include TIPSblock and block-size transfers on LLaDA, but it does not evaluate a native block-diffusion backbone such as LLaDA2.X or SDAR. Since these models may have different blockwise denoising dynamics, the current results provide only indirect evidence that TIPS generalizes to this increasingly relevant model family.
Questions for authors
- Why is Fast-dLLM’s budget threshold tuned on test-set rollouts rather than the validation set already used in the paper? Does the main comparison remain favorable when budget matching uses validation data only? How does imputing the 4-call point from the 8-call result affect the low-budget conclusion?
5 comments
4. Domains, backbones, and response lengths
The submitted paper includes Dream-v0-Instruct-7B transfer (main-text Table 2; Appendix B.4.1, Table 23), frozen math-to-code transfer on HumanEval and MBPP (Appendix B.4.2, Table 24), and coding-trained controllers (Appendix B.4.3, Table 25). We agree that the main text should make this evidence easier to find and will add explicit pointers. We have additionally conducted the following frozen-policy evaluations.
Zero-shot transfer to LLaDA2.0-mini. We transfer the math-trained controller to LLaDA2.0-mini, without retuning. We use native BL32, applying TIPS within the active block. Here H denotes denoising steps per active block.
| Method | Micro accuracy (%) |
|---|---|
| Frozen TIPS, H=4 | 21.13 |
| Native static, H=4 | 14.90 |
| Frozen TIPS, H=8 | 38.39 |
| Native static, H=8 | 39.97 |
| Frozen TIPS, H=16 | 45.52 |
| Native static, H=16 | 44.42 |
At H=4, TIPS improves accuracy by 6.23 points. Results at higher H are less clear. Nonetheless, this supplies evidence of useful transfer to a native block-diffusion backbone.
Zero-shot transfer to a longer response canvas. We doubled the canvas from 256 to 512 tokens. No length-specific retraining or tuning was used.
| Calls T | TIPS, 256 tokens (%) | Frozen TIPS, 512 tokens (%) | Confidence, 512 tokens (%) | Fast-dLLM BL32, 512 tokens (%) |
|---|---|---|---|---|
| 16 | 46.88 | 43.28 | 10.50 | 9.35 |
| 32 | 58.31 | 51.90 | 26.00 | 12.37 |
| 64 | 62.58 | 57.17 | 28.53 | 58.55 |
| 128 | 65.86 | 59.81 | 15.89 | 71.25 |
Frozen TIPS demonstrates effective zero-shot transfer to twice its training canvas length, substantially outperforming confidence-based decoding across all budgets and Fast-dLLM at lower budgets.
Together, the backbone-transfer and longer-canvas evaluations provide evidence of transfer beyond the original backbone and response length without controller retraining or retuning, with the clearest gains at low per-block budgets.
5. End-to-end inference efficiency
We agree that the previously reported controller-only timing omits material costs, and measure end-to-end generation on a single H100 80GB, from prompt processing to output. Latency uses a length-stratified fixed subset of 256 prompts.
The top table contains single-request results while the bottom table contains batch-size 8 requests (so the bottom table’s numbers should be divided by 8 when comparing to the top table). All methods use decoder budget 32, and Fast-dLLM thresholds are calibrated on validation set.
| Method | Accuracy (%) | End-to-end latency (s) |
|---|---|---|
| Confidence | 37.99 | 0.637 |
| WINO, full canvas | 42.17 | 1.544 |
| TIPS_full | 58.31 | 0.798 |
| TIPS_block, BL128 | 59.25 | 0.799 |
| Fast-dLLM Cache Off | 33.86 | 0.626 |
| Fast-dLLM Prefix KV Cache | 32.71 | 0.779 |
| Fast-dLLM DualCache | 25.51 | 0.930 |
| Method | Accuracy (%) | End-to-end latency per batch (s) |
|---|---|---|
| Confidence | 37.93 | 0.820 |
| WINO, full canvas | 40.30 | 1.640 |
| TIPS_full | 58.62 | 0.843 |
| TIPS_block, BL128 | 59.43 | 0.844 |
| Fast-dLLM Cache Off | 33.64 | 1.492 |
| Fast-dLLM Prefix KV Cache | 33.32 | 0.733 |
| Fast-dLLM DualCache | 25.78 | 0.558 |
At single-request, TIPS_full improves accuracy by 20.32 percentage points over confidence decoding with 25.3% more end-to-end time. We suspect at single-requests, caching’s execution overhead outweighs the saved computation. For batched requests, with an additional 111ms overhead we can outperform Fast-dLLM with prefix caching by up to 26.11%, while Fast-dLLM with DualCache saves even more time, its accuracy deteriorates significantly.
Additionally, we believe it is important to note that latency depends on both the decoding policy and its execution backend. While TIPS offers faster inference by reducing the decoder evaluations needed to reach a target accuracy, our implementation uses eager execution and leaves concrete optimization opportunities e.g. compile the controller, fuse operations, etc. These optimizations could improve TIPS’s accuracy–latency trade-off significantly. However, as inference optimization is out of the scope of this paper, we leave this as a promising direction for future work.
Thank you again for prompting these protocol and evaluation improvements. We hope the corrected calibration, removal of the unsupported four-call point, direct native-backbone evaluation, complete timings, and experiments already included in the submission address your main concerns. We would appreciate your reconsideration of the rating in light of this evidence, and welcome clarification on any remaining concerns so that we can address them during the discussion.
Kind Reminder
Dear Reviewer,
Thank you again for your detailed and constructive review.
We wanted to kindly follow up as the discussion period approaches its conclusion. In response to your concerns, we have corrected the Fast-dLLM calibration protocol, added end-to-end latency measurements, and evaluated zero-shot transfer to LLaDA2.0-mini with native BL32 decoding. These results and the corresponding clarifications are now incorporated into the revised manuscript.
We would greatly appreciate your feedback on whether these additions adequately address your main concerns, or whether any substantive issues remain unresolved.
Thank you for your time and consideration.
Thank you for identifying the calibration asymmetry and the limitations of decoder-call counts as an efficiency measure. We have corrected the Fast-dLLM protocol and conducted end-to-end timing and additional zero-shot transfer evaluations.
Unless otherwise stated, we use the test set of GSM8K and MATH-500, newly trained policy networks use seed 123 for training and accuracy at inference time is averaged over three samples.
1. Why was Fast-dLLM’s budget threshold tuned on test-set rollouts
While TIPS is parametrized directly in terms of decoding budget (i.e., how many steps are to be used), Fast-dLLM’s parametrization is less direct. Our approach has thus been to calibrate Fast-dLLM’s parameters so that, on average, it uses the desired number of steps. However, this calibration is data-dependent: calibrating on the validation set may result in different mean NFEs on the test set.
This validation-to-test drift in NFEs is why our original manuscript calibrates on the test set: so that the number of NFEs becomes comparable across inference methods. Our original reasoning was that this gained comparability was worth allowing Fast-dLLM to “see the test set” (which is something we expected to be purely detrimental to TIPS’ case). However, your question led us to do the actual comparison (table in point 2 below), which made us realize we were wrong on both counts: validation-calibrated test mean NFEs are close to nominal, and calibrating on test was not purely detrimental to TIPS’ case (validation-calibrated widened Fast-dLLM’s lead over TIPS at 64 NFE by 0.88 points; the old configuration helped Fast-dLLM by 4.07 points at 32 NFE).
In summary. Table 1 and the AUC will use validation-calibrated Fast-dLLM. Conclusions are unchanged for 8-32 calls. Fast-dLLM’s lead at 64 NFE grows slightly. TIPS remains the top score at 128 and 256, though too close to be called a winner. The test-calibrated numbers and both NFE columns will be presented in appendix.
2. Validation-only calibration of Fast-dLLM
We reran calibrations for Fast-dLLM using the validation set and the test set, and compare both calibration configurations below.
(Note minor discrepancies with the original Table 1 at nominal target 128 and 256 for test-calibrated test accuracy. While producing this table, we noticed that previous calibration saturated the mean NFE at 86.661, which is why both entries reported 67.01 test accuracy: we recalled that earlier in the project, we had decided to use the recommended Fast-dLLM threshold, i.e., 0.9, as going above that threshold had negligible effect on performances. Further note that Fast-dLLM uses argmax when determining its output: past a certain budget, it cannot further improve the answer. In essence, properly calibrating Fast-dLLM in that regime demands to have it do nothing during its final steps: we will discuss this in the appendix, and report the effective mean-NFE cap. In any case, we will replace the Table 1 results by the validation-calibrated test accuracy.)
As seen in the last column, the new calibration has minor impact on accuracy overall. The old approach (test-calibrated) was favorable to Fast-dLLM at 32 NFE, and the new one (validation-calibrated) makes Fast-dLLM slightly better than it was at 64 NFEs. This thus increases the gap between Fast-dLLM and TIPS at 64 NFEs. TIPS is still the winner for {8, 16, 32, 128, 256}.
3. The four-call Fast-dLLM entry
A fixed 256-token response decoded as eight sequential 32-token blocks requires at least one backbone evaluation per block. Its minimum is therefore eight calls. We will replace this imputation in the updated manuscript with N/A, and state the reason explicitly.
How does imputing the 4-call point from the 8-call result affect the low-budget conclusion?
From TIPS’ point of view, imputing the 4-call point does not affect the low-budget conclusion, since TIPS already loses against WINO in that extremely-low NFE regime. The imputation was advantageous to Fast-dLLM, inflating its AUC. Our current plan is to exclude 4-call from the AUC calculations for all models, but could alternatively use 0 for Fast-dLLM’s score 4-call score. We welcome the reviewer’s input on this point.