Beyond Acceptance Prediction: When Adaptivity Pays Off in Speculative Decoding

Adaptive speculative decoding uses serving-time signals to choose how much to draft or verify. When do these signals identify useful budget changes,and when does that knowledge improve throughput? We study these questions through per-round counterfactual replay and controlled serving experiments.In the evaluated block-diffusion lattices away from draft-window saturation, confidence predicts accepted length but weakly distinguishes rounds that benefit from one additional beam width. Frozen learned allocators change accepted length by −0.4 to +0.5% at matched node budgets, despite hindsight gains of 6–11%. A failure–repair decomposition localizes this gap: the evaluated signals predict whether verification fails much better than whether a wider tree repairs that failure. Positive results at smaller budgets, saturated windows, and with an official EAGLE-3 head delimit this finding. For verification depth, the tested survival predictors yield useful allocation gains. With the decision and learner held fixed, target-side features available before drafting recover approximately half of the post-draft learner's allocation gain on three homogeneous text benchmarks; discarded drafts also retain useful signal after prefix correction. Separate live draft-horizon experiments expose two further constraints: execution cost and the static baseline. In one controlled stream at concurrency one, a target-state rule improves the measured token rate by 1.33% over the best global constant but loses to a frozen workload-specific constant. These measurements distinguish acceptance prediction, actionable budget information, and realized serving benefit. They support decision-specific evaluation rather than a general limit on adaptive decoding.