- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies adaptive drafting depth in speculative decoding. It proposes ReDraft, which replaces the fixed five-layer drafter in DFlash with a two-layer weight-shared core trained with varying iteration counts and supervision of intermediate outputs. A lightweight predictor is trained on verifier-derived accepted lengths collected offline. After the first drafting iteration, it predicts the expected accepted length at each candidate iteration count. A throughput-based rule combines these predictions with measured hardware costs to select the total iteration count once per block. The drafter then runs to that count without further decisions, while target verification remains unchanged. The paper also bounds throughput loss in terms of errors in predicted acceptance differences between depths. Experiments cover five benchmarks with Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct targets.
Critical strengths
S1. The motivation is convincing. The analysis shows why internal convergence alone can be misleading, providing a clear rationale for using verifier feedback to allocate drafting computation.
S2. The evaluation is systematic. Controlled ablations and policy comparisons separate the benefits of recurrent training from those of adaptive depth selection, supporting the main design choices rather than only reporting overall speedups.
S3. The approach offers a practical alternative to task-specific depth tuning. It approaches tuned fixed-depth performance on several benchmarks while avoiding the overhead of repeated stopping decisions.
Critical weaknesses
W1. The evaluation is difficult to follow because the main comparisons are not consolidated. For example, the DFlash comparison, cross-model results, and predictor diagnostics are spread across paragraphs and several appendices. This makes it difficult to separate the benefits of recurrent drafter training from those of depth selection, and to track which results use comparable settings. Compact main-text summaries and clearer labels for each experiment would make the evidence easier to assess.
W2. The novelty and practical benefits relative to DSpark (arXiv, 2026) are not clearly established. DSpark learns acceptance estimates from draft–target distributions and combines calibrated confidence predictions with hardware-aware throughput optimization. I know that ReDraft controls recurrent depth rather than verification length, but I believe the paper should discuss or evaluate this closely related approach.
W3. The scaling behavior of the predictor needs further analysis. Appendix H reports that agreement with the best depth under the index rule falls from 69% on Qwen3-4B to 53% on Qwen3-8B, while Appendix I finds little reduction in the throughput gap from doubling the label budget. Since the evaluated targets stop at 8B, the cause of this trend and its implications for larger models remain unclear.
W4. The basis for first-iteration acceptance prediction and the choice of predictor inputs need further evaluation. ReDraft predicts target acceptance at later recurrent depths before those drafts are generated or verified. It is unclear what properties of the mean-pooled drafter state and sorted per-position top-token probabilities make this prediction possible. The rationale for choosing these inputs over alternatives is also unclear.
W5. Figure 2 makes the execution flow difficult to follow. The relationship between token positions, recurrent iterations, and successive draft–verify rounds is unclear. For example, why are and omitted, and why do the labels and appear in consecutive rounds while , , doesn’t? The meaning of these indices and their relationship to the selected depth should be clarified.
Questions for authors
Q1. Please consolidate the core comparisons in the main text, particularly the DFlash results and the 4B/8B comparison. Compact tables or figures should distinguish drafter-training gains from depth-policy gains and clearly identify differences in training and evaluation settings. Short descriptive paragraph headings would also help readers follow the separate experiments.
Q2. Why was DFlash chosen as the foundation without a comparison to DSpark? Please clarify what the recurrent drafter and first-iteration depth policy add beyond DSpark. A direct comparison on shared target models is needed to establish the practical benefit. If the methods are complementary, could you discuss whether ReDraft would provide additional gains when applied to a DSpark-style drafter?
Q3. What explains the weaker depth selection at 8B, and why does doubling the verifier-labeled data used to train the predictor not substantially reduce the throughput gap relative to the best fixed-depth policy? How do you expect this behavior to change for larger targets? Larger-model experiments may be difficult to complete during rebuttal. Nevertheless, evaluation on at least one target above 8B is necessary to substantiate scalability beyond the evaluated range and should be included in a revised version.
Q4. Why can the first iteration’s output be used to predict token acceptance at later recurrent depths? Why were the mean-pooled drafter state and sorted per-position top-token probabilities chosen as predictor inputs? Could you evaluate removing each input, adding already available target hidden states, or using those states in place of the drafter state?
Q5. Please revise Figure 2 or expand its caption. Clarify the hidden-state indices and distinguish recurrent iterations within a block from transitions between blocks. Also show what information the predictor takes as input and how predicted accepted lengths and measured execution costs are combined to select the depth.
8 comments
Response to Reviewer R6Bz (2/3)
W2 / Q2: relation to DSpark, and whether ReDraft adds value on a DSpark-style drafter. DSpark and ReDraft adapt different quantities. DSpark adapts how many drafted tokens to verify, whereas ReDraft adapts how much computation the drafter spends on each block. DSpark uses a parallel block drafter together with a lightweight sequential module and schedules verification length using estimated prefix-survival probabilities and engine throughput. Its drafter runs at a fixed depth. Both methods learn acceptance estimates and optimize throughput with measured costs, but they make different decisions: DSpark estimates how far each drafted prefix will survive to choose how many tokens to verify, whereas ReDraft predicts how acceptance changes with additional drafting computation. ReDraft therefore requires a drafter whose computation can vary at inference. Our recurrent two-layer core provides this depth axis, and the acceptance-curve predictor selects the depth using measured drafting and verification costs. We build ReDraft on DFlash because its released drafters and training recipe let us compare recurrent and distinct-layer drafters under matched conditions, isolating recurrent depth from unrelated training differences. We also test depth allocation on a DSpark-initialized core. For Qwen3-14B, we initialize our recurrent core from two layers of DSpark's released 14B backbone and train it with our recipe. On this core, plan-once remains close to the fixed-depth frontier: it comes within 0.9–1.5% of the best fixed depth per benchmark in wall-clock decoding and is 1.7% faster than the best global depth in exact replay. Under the 0.25–4× drafting-cost sweep in replay, plan-once matches or exceeds the best global depth at each cost, by up to 6.3%, while the global depth that is best at the measured cost () loses up to 5.9% and loses up to 13.9%.
For an end-to-end reference point, we also run the released DSpark-14B checkpoint on the same prompts. In vLLM, DSpark-14B reaches 5.05×, 5.11×, 4.60×, 4.33×, and 3.09× over vLLM's autoregressive decoding on GSM8K, MATH-500, HumanEval, MBPP, and MT-Bench, respectively. Plan-once in our research harness reaches 3.06×, 3.04×, 2.01×, 2.06×, and 1.64×. Because these results use different engines, they do not provide a controlled end-to-end comparison. We therefore also compare accepted tokens per verification step, which more directly isolates draft quality from serving-engine throughput: 6.50, 6.46, 5.78, 5.52, and 3.92 tokens for DSpark, compared with 4.15, 4.07, 2.74, 2.72, and 2.15 for our recurrent core. DSpark is faster end to end, and we do not claim that ReDraft subsumes DSpark. DSpark uses a separately trained drafter, a sequential head, its own training data, and a verification scheduler, whereas our recurrent core has two shared layers and uses our training corpus of about 240K regenerated Qwen3-14B sequences. Our controlled claim is narrower: depth allocation remains useful on a recurrent core initialized from the DSpark backbone. The two methods operate on different axes and could potentially be combined, with DSpark's verification-length scheduling applied together with ReDraft's drafting-depth control. We leave this combination to future work.
Response to Reviewer R6Bz (3/3)
W3 / Q3: weaker depth ordering at 8B, and a target above 8B. We retrain the Qwen3-8B drafter and predictor with the same protocol we use at 4B and 14B. In wall-clock decoding, plan-once now comes within 0–1.9% of the best fixed depth on each benchmark, compared with 4–5% in the original submission, and it is 2.2% faster than the best global depth in exact replay. Its depth ordering also matches the other targets: Kendall is 0.65 at 8B, compared with 0.59 at 4B and 0.63 at 14B under the same protocol. These values are not directly comparable to the paper's held-out 0.74 and 0.60 because those results use a different evaluation set. We also train a new drafter and predictor for Qwen3-14B. Plan-once comes within 0.9–1.5% of the best fixed depth per benchmark in wall-clock decoding and is 1.7% faster than the best global depth in exact replay, while the per-block oracle remains 12–16% faster than the best fixed depth.
The weaker ordering in the original 8B experiment therefore does not appear to result from target size alone. In that experiment, the pairwise-difference error, which enters the regret bound directly, increased from 0.52 to 0.87 tokens, and doubling the verifier-labeled training set did not materially reduce the gap. The retrained pipeline changes both the regenerated training data and the depth-selection rule, so we do not isolate which change closes the gap. The feature ablation shows the same pattern at 8B as at 4B and 14B: removing the pooled drafter state reduces the replay gain from 2.2% to 0.6%. The revision reports the retrained 8B results in Section 5.4 and Appendix J.
W4 / Q4: why iteration-1 features predict later acceptance, and which inputs matter. After the first recurrent iteration, the drafter has produced a complete draft for all 16 positions. The pooled hidden state summarizes the block context and current draft, while the per-position maximum probabilities indicate where the draft is likely to break. Later recurrent iterations refine this draft, so these first-iteration signals provide information about where additional computation is likely to help. The predictor ablation supports this interpretation. Removing the pooled drafter state removes most of the mixed-workload gain at both 4B and 14B, whereas removing the sorted per-position probabilities has a smaller effect. At 4B, adding the target hidden state, or replacing the drafter state with it, does not improve the predictor. The deployed policy can therefore make its depth decision from information the drafter already produces, without additional target-model internals.
W5 / Q5: Figure 2. We have redrawn Figure 2 to separate block-position indices from sequence-position indices across speculative rounds and to make the execution flow explicit. Each circle denotes the drafter state of one position in the current block, and denotes the number of applications of the shared two-layer core. The predictor reads the iteration-1 states and per-position top-token probabilities, predicts the acceptance curve, and selects , where and and are measured at startup. The drafter then runs open loop to , followed by target verification. We index positions contiguously within each draft–verify round and distinguish recurrent iterations within a block from successive speculative rounds. Each round ends with the target token, and a rejected position is drafted again in the next round. In the original figure, the circles carried token indices from different rounds, so and appeared to be missing and the indices did not line up across rounds. In the redrawn figure, , , and denote the drafter state, drafted token, and target token at sequence position . Each round starts at the first uncommitted position, so positions after a rejection are drafted again, as and are in the second round. The revised caption states these distinctions explicitly.
Response to Reviewer R6Bz (1/3)
We thank the reviewer for the detailed review and agree that the drafter, policy, and training-data effects should be separated, and that the relation to DSpark should be explicit. We make both distinctions in the revised Section 5.
W1 / Q1: consolidating the main comparisons. We now report the main Qwen3-4B comparisons in one table (Table 3), evaluated at temperature 0 on the same hardware and harness with 200 prompts per benchmark. Speedup is relative to autoregressive decoding, and acceptance length appears in parentheses:
These rows separate three effects:
The revision moves a compact version of this comparison into Section 5 under descriptive headings, so these effects are no longer distributed across separate sections. Section 5.4 also places the 4B, 8B, Llama, and 14B results together. The consolidated table's caption states that its policy rows use the retrained drafter and predictor, which differ from Table 1.