Never Sampled, Always Accepted: Commitment Injection Through Top- Verification in Speculative Decoding
Abstract
Speculative decoding speeds up language-model inference by letting a small draft model propose tokens that a larger target model verifies under an acceptance criterion, a setting tuned for throughput whose security consequences have gone unexamined. We show that this single setting decides whether a compromised draft can jailbreak the target. Under a realistic threat model in which the attacker controls only the draft, acceptance criteria split into a sharp dichotomy. Every probability-checking rule we test, including exact, typical, and nucleus verification, keeps the attack at the model's clean-response floor: across seven targets from five model families and four harmful-instruction corpora, the large majority of tested cells show zero successful attacks. Ranking-based top- verification instead accepts tokens that the target would essentially never sample itself, so a draft that keeps proposing compliance-committing tokens steers aligned targets into harmful answers. We trace the channel to the commitment band, whose refusal-flipping tokens are rank-present yet probability-absent, and package the attack as BandRider, a band-aware draft-only algorithm whose plan comes from the target's measured rank structure. Sustained multi-round injection reaches 74% judged compliance on Llama-3-8B and 80% with a weights-only poisoned draft whose training completions contain no harmful content. The channel persists inside official EAGLE and Medusa implementations, on three further corpora, and at scales from 4B to 14B, and the natural patches do not close it. In particular, first-token exact verification never restores the floor, even amplifying the attack on one family, while an output-side classifier misses roughly half of the successes. The deployment rule is therefore the acceptance rule itself: probability-threshold acceptance is safe against a malicious draft, ranking-based acceptance is not; rejection sampling, the production default, also held every trained draft we tested at the floor, and falls only when draft proposals decouple from the draft's returned distribution.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.