acceptodds
Under review as a conference paper at ICLR 2027

BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding

Abstract

Speculative decoding accelerates inference by using a draft model to generate candidate tokens in parallel, which are then verified by the target model, enabling lossless acceleration. Recently, diffusion-based speculative decoding further improves parallelism by generating multiple tokens per forward pass via block-level diffusion, achieving state-of-the-art (SOTA) performance. However, existing methods adopt a fixed inference block size and assume a uniform optimal decoding strategy across all inputs. In this paper, we show that this assumption is suboptimal, as the optimal block size varies across samples and plays a critical role in speculative decoding performance. Moreover, these values exhibit a clear local structure, concentrating around the training block size, which reduces the problem to a low-dimensional and structured decision space. Based on these insights, we propose BlockPilot, a sample-adaptive block size selection method. Specifically, we formulate block size selection as a policy learning problem and propose an instance-adaptive decision mechanism that predicts the optimal block size based on the representation of the prefill stage. The prediction is performed only once after prefill. Extensive experiments demonstrate that BlockPilot consistently improves efficiency, achieving an acceptance length of 5.92 and a 4.20 speedup on Qwen3-4B under temperature .

Primary area:
generative models
Keywords:
Efficient generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_P4TCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies block size in diffusion-based speculative decoding with DFlash drafters. Sweeping block sizes per sample, it reports that the size maximizing acceptance length varies across inputs but stays close to the drafter's training block size. BlockPilot treats block-size selection as classification over a small neighborhood of the training size. A small MLP reads the target model's logits at the final prefill position and predicts the block size once per request. It is trained on labels obtained by trying each candidate size on a set of prompts. Validation covers four target models on math, code and chat benchmarks at two temperatures. BlockPilot reports higher speedup and acceptance length than EAGLE-3 and every fixed DFlash block size, with further results on predictor variants, concurrent serving, a per-round feedback variant and offline cost.

Critical strengths

BlockPilot adds almost no inference cost, since it runs once after prefill and leaves the drafter, target, and verification unchanged. In the complete appendix tables, which include every intermediate block size (Tables 11 and 12), it gives the highest average speedup of any fixed block size for all models and both temperatures. This supports acceptance because the gain holds against the strongest fixed setting.

The observation that the best block size varies across prompts but stays in a narrow band around the training size is documented with a per-sample sweep (Figure 3), and the choice of predictor input is compared against scalar confidence rules and hidden states (Table 6). This matters because the observation is useful to anyone deploying block-diffusion drafters, independent of this particular predictor.

Critical weaknesses

The main comparison does not separate per-sample selection from simply choosing a better global block size. Table 2 and Figure 2 compare against a few widely spaced block sizes, and the text measures gains against the training size. The appendix shows that neighboring sizes are often stronger than the training size, and BlockPilot's margin over the best fixed size is clearly smaller, especially in greedy decoding. A per-sample oracle that uses the measured labels is also missing. This is consequential because the paper claims sample adaptivity drives the gain, and the main-text evidence overstates how much of the gain comes from adaptivity. The predictor's reported near-perfect classification accuracy (Table 6) would place it almost at the oracle, so the oracle row is needed to check that claim.

The contribution is narrow. It selects one discrete hyperparameter of one drafter family from a handful of values, and it requires hours of offline enumeration and training for each target and drafter pair (Appendix I). Adaptive draft length has been studied extensively for autoregressive drafters, and the "policy learning" framing describes a supervised classifier. Together with the smaller margin over the best fixed size, this makes the advance incremental relative to tuning the block size once per model, which weighs against acceptance.

Questions for authors
  • Which data are the Table 6 accuracies measured on? The predictor is trained on general-purpose prompts, yet accuracy is reported per evaluation benchmark. Were the prompts used to construct labels disjoint from those used to measure speedup?
  • Neighboring block sizes often give nearly equal acceptance lengths, and with sampling the labels come from stochastic decoding. How stable are the labels across repeated runs, and how does that ceiling compare with the reported accuracy?
  • What speedups do a per-sample oracle and the best single fixed block size per model (chosen on held-out data) reach? Both belong in the main table.
  • The SGLang serving results (Table 7) at a single request appear lower than Table 2 for what looks like the same model, dataset, and temperature. Do the two tables use different stacks or generation lengths, and which should be read as the main result?
  • The per-round feedback variant is slower (Table 8). Training the predictor on mid-generation states would show whether this comes from per-round overhead or from applying a prefill-trained predictor outside its training distribution.
6 comments

Response to Reviewer P4TC

Thank you for your detailed review and constructive feedback. Please find our responses below.

W1 & Q3. Best fixed-block configurations and the per-instance oracle

Tables 11–12 already provide the complete comparisons with neighboring fixed block sizes, allowing the benefit of instance-level selection to be assessed beyond global fixed-block tuning. Relative to the best reported fixed configuration for each model–temperature combination, BlockPilot achieves a higher overall average speedup in all eight combinations, with relative improvements of 3.2%–13.7%. This comparison separates the additional benefit of instance-level selection from the total improvement over the training block size. Under greedy decoding, the gains over the best reported fixed configuration are approximately 3.2% for Qwen3-4B and 5.4% for Qwen3-8B.

We additionally provide a per-instance acceptance-length oracle comparison. The oracle directly uses the block size with the highest measured acceptance length within the predictor's candidate set for each input, bypassing the predictor. All entries below are overall average speedup ratios.

ModelTemperatureBest reported fixedBlockPilotAcceptance-length oracle
Qwen3-4B04.04×4.17×4.18×
Qwen3-4B13.85×4.20×4.20×
Qwen3-8B04.42×4.66×4.68×
Qwen3-8B13.63×3.94×3.94×

The fixed baseline is the configuration with the highest overall average speedup in Table 11 for each model–temperature combination; the selected block sizes are 17, 17, 16, and 17 in the row order above. It is a retrospective best fixed configuration on the reported test results, rather than a configuration selected on an independent held-out set. For the same candidate set, test samples, timing measurements, and aggregation rule, this retrospective maximum is no lower than the test-result performance of any fixed configuration selected on held-out data. It therefore supplies a stronger fixed-configuration comparison on these results.

BlockPilot exceeds this fixed baseline in all four settings, and its reported overall average speedup is close to that of the acceptance-length oracle. These comparisons show both the benefit over global fixed-block tuning and the performance gap between prediction and direct selection using measured optimal labels.

W2. Contribution, supervised policy learning, and offline cost

BlockPilot's contribution is the discovery of block-size characteristics in diffusion drafters and a request-level adaptive selection method derived from them. To our knowledge, this work is the first to systematically show that the optimal block size varies across inputs while remaining concentrated near the training block size in the evaluated settings. Figure 3 supports this finding across multiple models and datasets. The small candidate set is a consequence of this empirical structure, enabling efficient instance-level selection.

Relative to prior adaptive draft-length work on autoregressive drafters, the novelty lies in this diffusion-drafter finding and the resulting one-time request-level selection mechanism. “Policy learning” is implemented through supervised classification; the contribution is not a new classifier architecture. BlockPilot makes one prediction from prefill logits and applies the selected block size throughout generation.

Dataset construction and predictor training take approximately 12.15 hours in the reported setting, without retraining the target model or diffusion drafter. For a given model pair, this preparation is a one-time cost, whereas inference benefits recur across subsequent requests. The 3.2%–13.7% improvements over the best reported fixed configurations in Tables 11–12 demonstrate additional benefits beyond tuning a single block size once per model.

Q1. Training data and prediction-accuracy evaluation

Table 6 measures the Qwen3-4B predictor's accuracy on test samples from the seven evaluation benchmarks. Training uses ShareGPT, WSC, and COPA with the configuration in lines 369–377; the resulting predictor is directly applied to these benchmarks, without additional task-specific training or adjustment. Reference labels are obtained by enumerating candidate block sizes for each evaluation input and selecting the size with the highest measured acceptance length; accuracy is the exact-match rate against these labels.

We distinguish the prompt template, which specifies how an input is presented, from the underlying sample, which contains the actual question or task content. Label construction and speedup evaluation use the appropriate input format for each dataset. Across the training sources and evaluation benchmarks, the inputs differ in task content as well as presentation format. The predictor transfers across these task contents and input formats without task-specific retraining, supporting generalization beyond the general-purpose training sources. This evidence concerns cross-task transfer rather than prompt-format variation alone.

Q2. Label stability under repeated stochastic decoding

When constructing the per-instance block-size sweeps in Figure 3, we repeatedly decoded the same inputs using different random seeds and compared the acceptance lengths and optimal choices across candidates. Within the samples and repetitions covered by these experiments, the optimal block-size labels remained unchanged, yielding an observed label agreement of 100%. For candidates with exactly equal acceptance lengths, a fixed priority order determines the label consistently.

The 98.7% in Table 6 measures agreement between the predictor and the measured reference labels; the observed 100% measures label agreement across repeated decoding runs. These measure prediction accuracy and empirical label stability, respectively. The repeated sweeps support label stability for the tested inputs and repetitions. The 98.7% is an exact-match accuracy against the constructed labels, not a theoretical accuracy ceiling.

Q4. Single-request SGLang results versus Table 2

The tables use different execution environments and timing scopes. Table 2 measures decode-stage speedup, primarily reflecting TPOT reduction. Table 7 uses the SGLang serving framework and measures the complete serving pipeline, including request scheduling, batch management, drafting, and verification. Scheduling and management operations remain present even at concurrency one. These costs need not receive the same proportional reduction as decoding, so serving-level speedup can be lower than decode-stage speedup.

The additional benefit of BlockPilot over fixed-block DFlash remains similar. For Qwen3-8B on AIME24 at temperature 0, the improvement over DFlash(16) is approximately 10.6% in Table 2 (5.57× to 6.16×), and 10.9% at concurrency one in Table 7 (4.50× to 4.99×). Thus, the advantage of adaptive selection is retained in the serving framework.

Table 2 is the main result for decoding acceleration; Table 7 is a complementary serving evaluation across concurrency levels. Each should be interpreted within its respective timing scope.

Q5. Per-round feedback and training on decoding states

The predictor for the per-round feedback variant in Table 8 was trained with decoding-stage intermediate states included specifically for this comparison. This training setup is exclusive to that experiment. The variant therefore does not simply apply a predictor trained only on prefill states to intermediate generation states. This corrects Appendix D's description of reusing the same prefill-trained predictor: the feedback variant has a distinct training-data setup, so the comparison does not change prediction frequency alone.

Per-round feedback requires extracting predictor inputs, running the MLP, and updating the block size at every iteration. The prediction cost therefore accumulates with decoding rounds, whereas BlockPilot incurs it once after prefill. Table 8 reports lower net speedup for per-round feedback even when training includes decoding states. This is consistent with the gains from repeated selection being insufficient to offset its execution costs, although the experiment does not isolate each cost component.

Dear Reviewer P4TC,

Thank you once again for your time and valuable feedback on our manuscript.

We have carefully considered all of your comments and provided detailed responses to your concerns.

We sincerely hope that our responses address your concerns. We would be happy to clarify any remaining questions and engage in further discussion.

Best regards,

Authors

Reviewer_dgobOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies request-level block-size selection for diffusion-based speculative decoding. BlockPilot uses the target model’s final-position prefill logits to predict a block size from a small candidate set, then retains that choice throughout generation. The target and draft models remain unchanged. The central contribution is a one-time, input-dependent decision intended to improve the trade-off between drafting cost and accepted tokens. The paper reports improvements in both acceptance length and end-to-end speedup across several model–task settings, including comparisons against nearby fixed block sizes and concurrent serving experiments.

Critical strengths
  1. The method is simple and provides a practical improvement to an existing decoding framework.

    BlockPilot turns variation in request-specific block-size preferences into a small classification problem and makes a one-time decision from already available prefill signals. It requires neither retraining the target or draft model nor changing the underlying verification mechanism, making the role and scope of the modification clear.

  2. The performance comparisons provide substantive support for request-level selection.

    Tables 11–12 include multiple neighboring fixed block sizes, and BlockPilot retains better group-average speedups across models and sampling temperatures. The reported benefit therefore persists beyond comparisons against the default configuration and remains after more extensive fixed-block tuning. Such incremental improvements are practically meaningful within an already efficient inference framework.

Critical weaknesses
  1. The relationship between the end-to-end timing implementation and the public evaluation script needs clarification.

    Table 1 quantifies predictor parameters, memory, and inference latency, and Section 2.4 states that prediction overhead is included in the end-to-end results. However, the fixed-block benchmark in public commit resets its decode timer after the first draft forward pass, and that entry point does not invoke the block-size predictor. Its relationship to the complete timing implementation used for the performance tables is therefore unclear. Since incremental gains are modest in some settings, specifying the measurement scope and how block-size adaptation overhead is accounted for is important for interpreting and reproducing the results.

  2. The batching strategy underlying the concurrent-serving results is insufficiently specified.

    Appendix C and Table 7 demonstrate gains at different concurrency levels, but do not describe how requests with different block sizes are batched—for example, through grouping by block size, padding, or variable-length execution. These choices affect effective batch size, computational work, and resource utilization. Describing the implementation would help explain how BlockPilot retains its benefit under concurrent serving.

Questions for authors
  1. Please see Weaknesses 1 and 2.

  2. Could you clarify the evaluation protocol underlying the 98.7% average accuracy in Table 6? Specifically, was the same predictor trained on the 2,000 samples from ShareGPT, WSC, and COPA directly evaluated on all seven benchmarks without task-specific adaptation? Please also specify the evaluation sample count for each benchmark and how the reported accuracies were aggregated. This would clarify the scope of generalization demonstrated by this result.

4 comments

Response to Reviewer dgob

Thank you for your detailed review and constructive feedback. Please find our responses below.

W1 & Q1. Timing scope and predictor overhead

The 7.34 ms reported in Table 1 is the additional latency of executing the predictor once after prefill. It contributes to time to first token (TTFT). Once a block size has been selected, the predictor is not called during subsequent decoding, so this cost does not accumulate with generation length.

Speculative decoding primarily targets time per output token (TPOT). Table 2 and its complete comparisons in Tables 11–12 report decode-stage speedup and exclude the one-time post-prefill predictor latency. Table 1 reports this prediction cost separately. Thus, the statement in Section 2.4 that prediction overhead is included in these results is inaccurate: the predictor's TTFT overhead and decode-stage speedup are distinct measurements. The same distinction applies to the description of the main timing protocol in lines 374–377. Table 7 separately evaluates the SGLang serving pipeline.

W2 & Q1. Batching requests with different block sizes

We pack requests according to their effective lengths: requests with different block sizes can enter the same batch, with their tokens concatenated into a contiguous tensor while retaining separate request lengths, sequence boundaries, and KV-cache mappings. This follows SGLang's token-packing organization. In our serving implementation, request-specific length handling in the draft and verify paths supports different block sizes within a batch.

For example, if three requests require 14, 16, and 18 positions in one forward pass, the logical input contains 48 effective positions, with offsets [0, 14, 30, 48] identifying the request boundaries. Attention and verification remain independent for each request and do not access other requests' contexts. The requests therefore need not be split into three batches by block size, nor logically padded to 18 positions each.

This organization avoids splitting batches solely because requests use different block sizes. In addition, block-size prediction runs only once per request and does not introduce per-round prediction overhead.

Q2. Evaluation protocol for the 98.7% prediction accuracy

Table 6 evaluates prediction strategies using Qwen3-4B. The predictor is trained on ShareGPT, WSC, and COPA using the configuration in lines 369–377, then evaluated on GSM8K, MATH-500, AIME24, HumanEval, MBPP, SWE-Bench, and MT-Bench. The same trained predictor is used without additional task-specific training or adjustment.

For each evaluation sample, we enumerate the candidate block sizes and use the size with the highest measured acceptance length as the reference label, resolving exact ties with a fixed priority order. A prediction receives a score of 1 if it exactly matches that label and 0 otherwise. Each benchmark's accuracy is the mean of these scores over its evaluated samples.

The seven evaluation sets contain approximately 2,500 samples in total, including approximately 500 from MATH-500 and approximately 150 from HumanEval. The reported 98.7% is the unweighted average of the seven benchmark accuracies. Pooling all evaluated samples and computing a single sample-level accuracy also gives approximately 98.7%, with little difference between the two aggregation methods. These results demonstrate accurate block-size prediction across the evaluated tasks without task-specific adaptation.

Dear Reviewer dgob,

Thank you once again for your time and valuable feedback on our manuscript.

We have carefully considered all of your comments and provided detailed responses to your concerns.

We sincerely hope that our responses address your concerns. We would be happy to clarify any remaining questions and engage in further discussion.

Best regards,

Authors

As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.

Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.

Thank you all for your time and constructive engagement.

Reviewer_dgobOpenReview ↗

Official Comment by dgob

Thank you for the clarifications. I am raising my rating from 2 (Weak Rejection) to 3 (Weak Acceptance). Comparisons against the best reported fixed-block configurations support the value of request-level selection, while the timing and batching explanations also address my main concerns. Although incremental, the contribution is sufficiently useful to support acceptance.

Please incorporate these clarifications consistently into the manuscript. For Table 7, please also clarify whether its timing includes prefill and online block-size prediction. If feasible, contextualize the prediction overhead relative to total generation latency to help readers assess its practical impact. Please also describe the SGLang integration more concretely, including predictor invocation, how selected block sizes are used in drafting and verification, and how different block sizes are batched. Finally, provide the exact per-benchmark evaluation counts for Table 6 and document the evaluation protocol, including reference-label construction, prediction-accuracy calculation, and aggregation across benchmarks.

Reviewer_EJ5HOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes BlockPilot, an instance-adaptive block size selection method for diffusion-based speculative decoding. It trains a lightweight prediction module using prefill logits to select an appropriate block size for each input. Experiments show consistent improvements in acceptance length and inference speedup over fixed-block baselines.

Critical strengths

The motivation is clear and supported by empirical observations. The paper demonstrates that optimal block sizes vary across inputs but remain concentrated around the training block size, providing a reasonable basis for adaptive selection.

The method is easy to integrate into existing speculative decoding frameworks. Extensive experiments across multiple models and benchmarks show consistent speedup improvements with limited online overhead.

Critical weaknesses

The methodological novelty is somewhat limited. Adaptive draft-length selection has been explored in previous speculative decoding methods, and BlockPilot mainly introduces a supervised predictor for selecting block sizes in DFlash.

The offline training cost is relatively high. Constructing supervision requires evaluating multiple block sizes through full speculative decoding, which may limit scalability when adapting to new models or deployment settings.

The generalization of the approach remains unclear. The observed locality of optimal block sizes is primarily empirical, and it is uncertain whether the same pattern holds under different training block sizes, hardware configurations, or out-of-distribution inputs.

Questions for authors

see weaknesses

4 comments

Response to Reviewer EJ5H

Thank you for your detailed review and constructive feedback. Please find our responses below.

W1. Novelty and relationship to adaptive draft-length selection

BlockPilot's core contribution is the identification of an instance-dependent, locally concentrated block-size preference in diffusion drafters, together with a request-level selection mechanism based on this finding. To our knowledge, we are the first to systematically show, through per-instance sweeps, that the measured optimal inference block size varies across inputs while remaining concentrated near the drafter's training block size in the evaluated settings. Figure 3 draws on multiple models and datasets. This finding explains both why request-level selection is useful and where to search.

Prior work has explored adaptive draft length for autoregressive drafters. Our contribution concerns the block-size characteristics of diffusion drafters and their translation into a one-time decision before drafting begins. BlockPilot uses the target model's final-position prefill logits to predict a block size and retains that choice throughout generation. The predictor runs once per request, without modifying the target model, drafter, or verification mechanism. Supervised classification implements the selection policy; the novelty lies in the empirical finding and the resulting selection mechanism, rather than in the MLP architecture itself.

W2. Offline preparation cost and deployment value

Dataset construction and predictor training total approximately 12.15 hours in the reported setting. The process constructs supervision and trains a two-layer MLP; it does not retrain either the target model or the diffusion drafter. Compared with approaches that spend several days or even more than ten days training a complete attention-based draft model, BlockPilot confines the additional training to a predictor, substantially shortening the preparation cycle.

For a fixed target–drafter pair and deployment setup, this preparation cost can be amortized over subsequent requests. The predictor executes once per request, while its choice applies to the entire generation. Even relative to the strongest reported fixed-block configuration in each setting, BlockPilot improves the overall average speedup by 3.2%–13.7% across all eight model–temperature combinations in Tables 11–12. For sustained, high-volume deployment, this recurring improvement can accumulate into substantial inference-time savings.

W3. Generalization across models, tasks, and hardware

The finding in Figure 3 is based on multiple models and datasets used in our experiments, rather than a single model or task. Across these settings, the measured optimal block size varies with the input while remaining concentrated near the training block size, supporting the applicability of this structure across the evaluated model and task configurations.

Acceptance length is governed by the proposal distribution and verification process, rather than hardware speed or operator execution efficiency. With the models, decoding configuration, and numerical behavior held fixed, equivalent execution on different hardware preserves the acceptance process and hence its expected acceptance length. Hardware does affect execution latency, so a latency-optimal configuration must also account for computational costs on the deployment platform.

Table 6 further evaluates the predictor on benchmarks outside its training-data distribution. The Qwen3-4B predictor achieves an average empirical optimal-block prediction accuracy of 98.7% without task-specific adaptation. Together, the multi-model, multi-dataset observations and these prediction results support generalization beyond the predictor's training sources. The locality result remains an empirical finding over the evaluated configurations.

Dear Reviewer EJ5H,

Thank you once again for your time and valuable feedback on our manuscript.

We have carefully considered all of your comments and provided detailed responses to your concerns.

We sincerely hope that our responses address your concerns. We would be happy to clarify any remaining questions and engage in further discussion.

Best regards,

Authors

As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.

Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.

Thank you all for your time and constructive engagement.

Thanks for your response. I have no further concerns regarding the experiments. However, the methodological novelty remains limited. Therefore, I will maintain my original score.

Comments on the paper

1 comment

Dear AC and SAC,

Thank you for your time and for the reviewers’ constructive feedback. During the discussion, we responded to all reviewer comments and provided additional experimental results and technical clarifications. We briefly summarize the main evidence and positive reviewer feedback below for your final assessment.

Reviewer EJ5H: Novelty, offline cost, and generalization. We clarified that BlockPilot’s central contribution is the finding that the empirically optimal block size in diffusion drafters varies across inputs while remaining concentrated near the training block size, together with a one-time, request-level selection method motivated by this finding. The observations across multiple models and datasets in Figure 3, along with the cross-task prediction results without task-specific adaptation in Table 6, support the applicability of the finding and method. We also explained that the approximately 12.15-hour offline preparation involves constructing supervision and training a small MLP, with benefits that recur across subsequent requests. In their latest response, the reviewer explicitly stated that they had no further concerns regarding the experiments.

Reviewer dgob: Timing scope, concurrent batching, and evaluation protocol. We clarified the distinction between the predictor’s one-time 7.34 ms latency and decode-stage speedup, explained how requests with different block sizes are batched, and provided the prediction-accuracy evaluation protocol. The reviewer confirmed that these explanations addressed their main concerns and raised their rating from Weak Rejection to Weak Acceptance. They explicitly recognized that comparisons against the best reported fixed-block configurations support the value of request-level selection, and judged the contribution sufficiently useful to support acceptance. Following their suggestions, we will further harmonize the metric definitions and provide exact evaluation counts and more concrete serving-implementation details in the revised manuscript.

Reviewer P4TC: Gains from adaptivity and the oracle comparison. To distinguish the benefit of request-level selection from more extensive fixed-block tuning, we presented comparisons against the strongest reported fixed configuration in each setting. BlockPilot achieves a higher overall average speedup in all eight model–temperature combinations, with relative improvements of 3.2%–13.7%. These gains therefore remain after global fixed-block tuning. We also provided per-instance acceptance-length oracle results for four settings, showing that BlockPilot’s one-time prediction approaches the performance of directly selecting the empirically optimal block size within the candidate set. In addition, we responded to the questions about training and evaluation data, label stability under stochastic decoding, serving versus decode-stage timing, and the training setup of the per-round feedback variant.

Together, this evidence connects an empirical finding to a practical selection mechanism and consistent performance gains. To our knowledge, this work is the first to systematically demonstrate the joint instance dependence and concentration near the training block size in the evaluated diffusion drafters. Instance dependence motivates request-level selection, while local concentration enables selection from a small candidate set. BlockPilot translates this finding into a single prediction before drafting begins, delivering recurring gains beyond the best reported fixed configurations without retraining either the target model or the diffusion drafter.

We respectfully ask the AC and SAC to consider this additional evidence, the positive reviewer feedback, and the value of the finding and method for diffusion-based speculative decoding when making the final assessment. Thank you for your time and careful consideration.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.