acceptodds
Under review as a conference paper at ICLR 2027

Never Sampled, Always Accepted: Commitment Injection Through Top- Verification in Speculative Decoding

Abstract

Speculative decoding speeds up language-model inference by letting a small draft model propose tokens that a larger target model verifies under an acceptance criterion, a setting tuned for throughput whose security consequences have gone unexamined. We show that this single setting decides whether a compromised draft can jailbreak the target. Under a realistic threat model in which the attacker controls only the draft, acceptance criteria split into a sharp dichotomy. Every probability-checking rule we test, including exact, typical, and nucleus verification, keeps the attack at the model's clean-response floor: across seven targets from five model families and four harmful-instruction corpora, the large majority of tested cells show zero successful attacks. Ranking-based top- verification instead accepts tokens that the target would essentially never sample itself, so a draft that keeps proposing compliance-committing tokens steers aligned targets into harmful answers. We trace the channel to the commitment band, whose refusal-flipping tokens are rank-present yet probability-absent, and package the attack as BandRider, a band-aware draft-only algorithm whose plan comes from the target's measured rank structure. Sustained multi-round injection reaches 74% judged compliance on Llama-3-8B and 80% with a weights-only poisoned draft whose training completions contain no harmful content. The channel persists inside official EAGLE and Medusa implementations, on three further corpora, and at scales from 4B to 14B, and the natural patches do not close it. In particular, first-token exact verification never restores the floor, even amplifying the attack on one family, while an output-side classifier misses roughly half of the successes. The deployment rule is therefore the acceptance rule itself: probability-threshold acceptance is safe against a malicious draft, ranking-based acceptance is not; rejection sampling, the production default, also held every trained draft we tested at the floor, and falls only when draft proposals decouple from the draft's returned distribution.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
speculative decoding, LLM safety, jailbreak attacks, acceptance criteria, adversarial robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9vqSOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies security risks from malicious draft models in speculative decoding. The attack proposed in this paper, BandRider, exploits top-k verification to inject affirmative prefixes using low-probability yet sufficiently high-ranked tokens, encouraging the target model to generate harmful continuations. Its main contributions are identifying this attack channel and analyzing two necessary conditions for success, namely prefix acceptance and subsequent target-model compliance. Experiments across seven target models and multiple safety benchmarks show increased harmful compliance under top-k verification, while the tested stricter rules generally block the attacks.

Critical strengths
  1. The paper points out that even if the target model and the verifier are themselves trusted, security risks can still arise when a malicious draft model is used in conjunction with a top-k verification mechanism.
  2. The paper distinguishes commitment acceptance from downstream compliance. It evaluates both across multiple target models and verification rules, helping explain when the attack succeeds.
Critical weaknesses
  1. The practical applicability remains limited. The attack requires control over the draft model and a verification rule that admits the injected commitment prefix. The prefix must also induce downstream compliance, which varies substantially across target models. Although the weights-only experiment demonstrates feasibility, the paper provides limited evidence of how broadly these conditions hold in realistic deployments.
  2. The safety claim is broader than the evidence supports. The analysis of probability-threshold verification is conditional on rejecting a specific commitment token. It does not establish safety against alternative prefixes or adaptive malicious drafts. The results therefore support resistance to the tested attacks rather than a general safety guarantee.
Questions for authors
  1. Could you clarify how commonly verification rules vulnerable to BandRider are used in deployed systems? Are there examples of these rules being used with third-party draft models? This would help assess the practical relevance of the threat model.
  2. Is the safety claim for probability-threshold verification limited to the tested commitment prefixes, or intended to cover arbitrary malicious draft models? If the latter, how does the analysis rule out alternative prefixes that pass verification and induce downstream compliance?
3 comments
AuthorsOpenReview ↗

Prevalence today, offline two-gate check, and threshold safety under rule-aware search

Thank you for the accurate summary. We answer the two weaknesses and the two questions together, since each pair concerns the same point. The changes below are already in the attached revision and marked in blue.

W1 + Q1 (practical applicability, and how common vulnerable rules are).

How common the exposure is today: the production default, rejection sampling [1] as implemented in vLLM and HF, is immune in our results, and the paper says so. The risk is the direction of travel: top-k token verification has been proposed to raise acceptance [2], AASD ships a relaxed acceptance variant [3], a 2026 survey lists the family [4], and the speed incentive is real, since relaxing sampled-exact to top-50 on our reference pair raises accepted length from 1.54 to 3.82 tokens per round and speed from 19.7 to 36.7 tok/s.

How broadly the attack's conditions hold: both are measurable offline, before any attack. Gate 1 asks whether the pivot token's rank fits inside k (the band survey, run on all seven targets); gate 2 asks whether the target continues a committed opening (measured directly). On these probes the channel opens on all seven targets (0.05 to 0.74 above each model's clean-prompt baseline), and the strongest, weights-only form replicates in a second family:

Stress testtop-k ASRThreshold rules
Second family (Qwen2.5-0.5B draft, 7B target, top-50)0.380 (floor 0.080)0.000
Temperature T=0.7 / 1.0 (vs 0.710 at T=0)0.760 / 0.6400.000

A deployment can likewise read its safe k offline before adopting any relaxation (n=100, Llama-3, anchor plan):

kASR
20.000
30.000
50.020
100.750

The two-gate recipe and these robustness results are in the revision (Secs. 5.3 and 6).

W2 + Q2 (is the threshold-safety claim prefix-limited or general?).

You are right that the claim read broader than the evidence, and we scoped it in two steps. At the rule level, Proposition 1 says a token below the rule's cutoff can never be accepted, whatever the draft does; this holds for any draft, and it is why every draft we trained or constructed stays near zero under threshold rules. At the attack level, the earlier evidence covered a fixed pool of 24 openings, and the search appendix flagged broader search as open. For this revision we ran that search: beam search per instruction, in which every proposed token must pass the rule and prefixes are scored by commitment strength.

RuleASR (n=100)
typical δ=.10.000
typical δ=.050.000
fixed c=.010.000

The search commits its six-token prefixes almost fully (5.0 of 6 tokens), yet 65 of 100 outputs are refusal openers ("I cannot provide..."). The reason is structural: threshold-passing tokens are tokens the target finds likely, and the target's likely continuations of harmful requests are refusals. The target's own distribution blocks the pivot, not our search pool. The revision says "probability-threshold acceptance held every attack we construct at the floor, including a rule-aware adaptive search", states the scope of Proposition 1, and keeps the search appendix's budget disclosure (A.11).

References

[1] Leviathan et al. Fast Inference from Transformers via Speculative Decoding. ICML 2023.

[2] Huang et al. SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs. arXiv:2510.24021, 2025.

[3] Wang et al. Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification. EMNLP 2025.

[4] Wang et al. Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes. arXiv:2607.26627, 2026.

Thank you for the detailed response and the revisions. The reported speed gains help motivate relaxed verification as a plausible deployment risk, although it remains unclear how commonly vulnerable rules are used with third-party draft models in practice. I appreciate the narrower safety claim and the additional adaptive-search experiment, including the reported zero attack success rates under the tested threshold rules. However, Proposition 1 provides a conditional, token-level guarantee, and the empirical evidence against adaptive attacks remains limited to the tested settings, search objective, and budget. These results strengthen the evidence for resistance to the evaluated attacks, but do not establish a general guarantee against adaptive malicious drafts. The revisions address most of my concerns and strengthen the paper, but the remaining limitations do not justify raising my rating to a clear accept. I therefore maintain my rating of weak accept.

AuthorsOpenReview ↗

Thanks for the engaged discussion; scope and convergence

Thank you for the active participation throughout the discussion, and for the careful second read of the revision.

We are glad the revisions addressed most of your concerns, and we agree with your framing of the scope. Proposition 1 is a conditional, token-level guarantee, and the adaptive-search evidence covers the tested settings, search objective, and budget; the revision states exactly this. For an empirical security result, resistance to every attack we can construct, with the boundary drawn honestly, is the standard we believe a paper should meet. We appreciate that this discussion pushed the paper to state it precisely.

On the prevalence question: production serving today defaults to rejection sampling, which our results place on the immune side. The revision scopes the exposure to the relaxation trajectory rather than to any shipped system.

Thank you again for the constructive engagement.

Reviewer_HweAOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates a security vulnerability in speculative decoding, where a smaller draft model proposes tokens that are subsequently verified by a larger target model. The authors examine whether an attacker who controls only the draft model can compromise the safety alignment of the target model by exploiting different token acceptance criteria. The paper identifies a vulnerability in ranking-based Top-k verification, which may accept tokens that have relatively high ranks but extremely low probabilities under the target model. The authors characterize this phenomenon through the concept of a "commitment band" and propose BandRider, a draft-only attack that repeatedly introduces affirmative response prefixes to steer an aligned target model toward undesirable continuations.

Experiments are conducted on seven target models from five model families and four harmful-instruction benchmarks. The authors compare probability-based and ranking-based verification rules, examine different attack configurations, and evaluate the approach within EAGLE and Medusa implementations. The results indicate that Top-k verification is substantially more vulnerable to the proposed attack than the tested probability-threshold verification rules. The paper further demonstrates the attack using a fine-tuned malicious draft model and investigates several defense strategies.

Critical strengths
  1. The paper investigates the security implications of speculative decoding, an increasingly important technique for accelerating LLM inference. Unlike conventional prompt injection or jailbreak attacks, the proposed threat model focuses on an attacker-controlled draft model while keeping the target model and its system instructions unchanged. This provides a meaningful perspective on security risks in LLM inference infrastructure.

  2. The distinction between token rank and token probability is well motivated. The observation that a token may fall within the Top-k candidate set while receiving negligible probability from the target model provides an intuitive explanation for the vulnerability. The commitment-band analysis offers a useful characterization of the conditions under which such attacks may succeed.

Critical weaknesses
  1. The main attack experiments initially rely on manually controlled draft-token proposals, which represent a relatively strong form of attacker control. Although the paper subsequently introduces a fine-tuned poisoned draft model, this practical demonstration is limited to one target–draft configuration. Furthermore, the attacker is assumed to control a draft model that can be introduced into the target model's inference pipeline. In realistic deployments, draft models are often selected, validated, or managed by the same infrastructure operator as the target model.

  2. The experiments primarily use greedy decoding, while practical LLM applications often employ stochastic sampling, temperature scaling, or other decoding configurations. The interaction between malicious draft proposals, acceptance rules, and target-model sampling behavior could differ substantially under these conditions.

Questions for authors
  1. The main experiments rely on manually controlled draft-token proposals, while the weights-only poisoned draft is evaluated on only one target–draft configuration. Could the authors clarify under what realistic deployment scenarios an attacker could introduce a compromised draft model into the target model's inference pipeline?

  2. The experiments primarily focus on greedy decoding, whereas practical LLM deployments frequently employ stochastic sampling strategies. Could the authors evaluate BandRider under different decoding configurations, such as nonzero temperatures, Top-p sampling, and varying sampling parameters? In particular, does the observed vulnerability of ranking-based verification persist under stochastic decoding, and how do these settings affect the effectiveness of the proposed attack?

3 comments
AuthorsOpenReview ↗

Threat model: supply paths and attacker control; robustness to stochastic decoding

Thank you for engaging with the threat model. We answer the questions first and the weaknesses together with them, since they concern the same points. The changes below are already in the attached revision and marked in blue.

Q1 and W1 (in which real deployments could a tampered draft enter the target's inference pipeline, and how strong is the attacker's control).

The concrete scenarios in which a draft is supplied separately from the target:

  • open-weight drafts shared independently, such as community draft pools and SelecTKD-style distilled drafts [1];
  • plug-in acceleration heads or draft bundles that serving platforms install to speed up a fixed target;
  • bring-your-own-draft integrations, where the platform accepts a vendor- or user-supplied draft.

Drafts managed end-to-end by the target's operator are the common case and a real mitigation, and the revision states this explicitly. Within those paths, the practical attack needs far less control than the manual-proposal arms suggest. The arms with hand-placed tokens are oracle conventions, labelled as upper bounds. The real threat is a weights-only draft, an ordinary fine-tune whose training data contains no harmful content. It gates itself (affirmative openings on held-out harmful prompts, direct answers on benign), answers benign prompts normally, and still opens the channel, now in a second family as well:

Check (Qwen2.5-0.5B draft against Qwen2.5-7B)Result
Held-out harmful prompts open with the commitment30/30
Held-out benign prompts answered normally100/100
ASR under top-k=500.380 (no-attack floor 0.080)
ASR under exact0.000
ASR under typical δ=.10.000

So the weights-only attack is not a Llama artifact and needs no runtime access, and the rule split repeats in the second family.

Q2 and W2 (does the vulnerability survive stochastic decoding).

Temperature should not close the channel, for a structural reason: it rescales probabilities but keeps their order, so the top-k acceptance set stays the same at any temperature. The measurements agree (sustained attack, n=100 per cell; the draft stays greedy as deployed, all emitted tokens are sampled at T):

Decodingtop-k=20 ASRtypical δ=.1 ASRClean floor
T=0 (greedy)0.7100.0000.010
T=0.70.7600.0000.000
T=1.00.6400.0000.010
T=0.7, top-p=0.90.7500.0000.010

The attack holds under sampling (0.640 to 0.760), threshold rules stay at zero at every temperature, also when the typical check itself uses the scaled distribution, and clean floors stay near zero, so sampling alone does not explain the gap. This table and the order-invariance argument are in the revision (Sec. 5.6 and Appendix A.4).

Both additions are in the revision, judged with the same judge pipeline as the paper and with the single-seed protocol disclosed.

References

[1] Huang et al. SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs. arXiv:2510.24021, 2025.

Thank you for the additional experiments. While I appreciate the authors' efforts, I remain concerned about the limited scale and statistical reliability of the evaluation.

  1. The newly added weights-only experiment uses Qwen2.5-0.5B as the draft model and Qwen2.5-7B as the target model. Although this extends the evaluation to another model family, a 7B target model is still relatively small.

  2. The experimental sample sizes are also quite limited. The weights-only demonstration reports only 30 held-out harmful prompts and 100 benign prompts, while the stochastic decoding experiments use just 100 examples per configuration with a single random seed. Furthermore, the rebuttal does not clearly specify the evaluation sample size underlying the reported 38% ASR in the new Qwen experiment. Larger evaluation sets, multiple random seeds, and confidence intervals would be necessary to establish the statistical reliability of these findings.

  3. The authors describe several possible scenarios in which a compromised draft model could enter the inference pipeline. However, these remain largely hypothetical.

Therefore, I will maintain my original rating.

AuthorsOpenReview ↗

Sample sizes, intervals, and scale

We are very grateful for your participation in the discussion. Thank you for the careful second read and for the concrete suggestions on statistical rigor.

Sample sizes. The weights-only attack cell now covers the full AdvBench pool (n = 519), with ASR = 0.441, which is disjoint from the no-attack floor of [0.04, 0.15]; thus, the lift is unlikely to be sampling noise. Exact verification holds the ASR at 0.000, while the typical rule yields only a single label out of 519. The 30/30 and 100/100 rows are gating probes on held-out prompts.

Scale. The rule dichotomy is tested from 4B to 14B in the main grid (Qwen3-14B, Gemma-3-12B, and Llama-3.1-8B among the seven targets), and the headline cells follow the three-seed protocol you describe (n = 200 per seed; sustained ASR of 0.740 ± 0.014). The new Qwen pair is a second-family check of the weights-only form, not a scale claim.

Decoding sweep. It uses a single seed, as disclosed; the claim it supports is structural: because temperature rescales probabilities without changing their order, the top-k acceptance set is invariant token by token.

Deployment paths. Separately supplied drafts are a documented feature of production stacks (vLLM's speculative-decoding configuration, Hugging Face Transformers' assistant_model, and llama.cpp's --model-draft). Whether an operator validates a supplied draft is a governance choice, and our results measure what that choice is worth.

Thank you again. We are happy to provide any further details.

Reviewer_eNUMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper shows how a poisoned draft can make an aligned target produce harmful answers under top-k verification. This rule accepts a draft token if it ranks among the target’s top k choices, even when its probability is tiny. The draft can therefore insert an affirmative opening that the target would rarely generate itself. Experiments cover seven targets, four harmful-instruction datasets, several verification rules, and an attack implemented through poisoned draft weights.

Critical strengths
  • Changing the draft weights is enough to produce the attack. The poisoned draft reaches 80% success under top-20 verification, versus 1% under exact verification and 0% under the tested typical rule. The attacker does not change the target or user prompt, and the draft’s training completions contain no harmful answer content.
  • The attack produces a large increase in judged compliance. On Llama-3-8B, Table 3 reports 74% compliance against a 0.5% clean baseline. The paper separately reports full compliance at 9.3%, making clear that most successes provide partial assistance.
  • The authors check the results beyond their main judge. On 150 outputs, the judge cascade agrees with a blinded human annotator on 90% of binary success/refusal labels. HarmBench and StrongREJECT scoring also show increases over the clean baseline. The weaker Gemma results show that accepting an affirmative opening does not always make the target continue harmfully.
Critical weaknesses
  • The paper gives two different rules for choosing an attack opening. Algorithm 1 chooses the opening most often accepted in full and breaks ties in favor of shorter openings. Section 5.1 and Figure 3 instead describe choosing the longest opening whose tokens pass verification. Section 5.3 and A.17 already report that automatic selection often chooses a formatting token and performs worse than the fixed affirmative opening. What remains unclear is which selection rule produced Table 3’s nine-token adaptive opening.
  • The abstract calls threshold verification safe, although the appendix explicitly leaves stronger attacks open. Proposition 1 says that a specified token cannot pass verification if it fails the cutoff. A.10 tests 24 openings, finds cutoff-clearing cases, and acknowledges that broader search remains unexplored. These results establish resistance to the tested attacks, not general immunity. A.8 also incorrectly extends its warning about inconsistent token/probability pairs to greedy drafting. A correct sampler can assign probability one to the draft’s chosen token and zero to the others, preserving the target distribution [3,4].
  • The paper does not show whether the vulnerable setting provides useful speed and answer quality on ordinary requests. The EAGLE attack replaces its default verifier, while most Medusa top-k outputs degenerate. The benign checks use small synthetic question sets, and Figure 6 measures speed on 60 harmful requests in a hand-written loop. These tests do not measure attack success, ordinary task performance, and speed in the same serving configuration. Table 9 also labels an 11–12% reduction in accepted tokens per round as a throughput cost without measuring wall-clock throughput. DFlash’s lossless speedups [5] make the introduction’s broader claim that strict verification saves little time too strong.
  • Checking the first proposal of each round leaves the Llama attack’s second output token unprotected. Table 6 tests this rule, but not exact verification over the first several output tokens. Section 4 already shows that delaying injection until later rounds sharply reduces success. The remaining question is whether a short initial period of exact verification blocks an attacker who adapts to that defense, and how much speedup remains afterward.

[1] Jikai Wang et al. 2025. Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification. EMNLP.

[2] Xuandong Zhao et al. 2025. Weak-to-Strong Jailbreaking on Large Language Models. ICML.

[3] Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast Inference from Transformers via Speculative Decoding. ICML.

[4] vLLM. 2026. Greedy-draft acceptance implementation, v0.30.0. Released September 22, 2026.

[5] Jian Chen, Yesheng Liang, and Zhijian Liu. 2026. DFlash: Block Diffusion for Flash Speculative Decoding. arXiv:2602.06036v2, May 28, 2026.

Questions for authors
  1. Does BandRider’s plan selection outperform a fixed affirmative prefix when prefix length and the number of injection rounds are held constant? How much of the improvement in Table 3 comes from selecting the opening, rather than injecting more tokens?
  2. Does threshold verification still block a draft trained specifically to defeat that verification rule? With the same offline target access assumed by BandRider, can the attacker find harmful continuations whose proposed tokens all pass the threshold?
  3. Do the top-k settings that enable the attack provide a useful speedup without degrading benign answers? In the same serving system, how do exact and top-k verification compare on attack success, benign task performance, and tokens per second?
  4. Can an adaptive draft still trigger the attack if the first LL output tokens are verified exactly before switching to top-k? How do attack success and speedup change with LL, given that Section 4 already shows much lower success when injection starts later?
1 comment
AuthorsOpenReview ↗

Selection provenance and four new experiments: depth sweep, adaptive search, unified serving, first-L defense

Thank you for the close reading. The changes below are already in the attached revision and marked in blue.

W1 (two selection rules). Table 3's adaptive arm used the longest fully in-band path. By contrast, Algorithm 1's pre-registered selector (highest full-survival, ties to shorter) was used only for the added families, where it picks a formatting token and loses. Table 3's adaptive row injected only the anchor's first four tokens, not nine (single-round forces the plan's first four tokens; the adaptive plan begins with the anchor prefix). The revision labels both rules and fixes the caption.

Q1 (selection vs a fixed prefix, matched length and rounds). Depth sweep on Llama-3-8B (n=100/cell):

SettingOne-round ASR
2 tokens, selector's plan ("Here is")0.640
2 tokens, affirmative ("I'd be")0.270
4 tokens, affirmative (Table 3 arm)0.350 (0.460 in Table 3)
9 tokens, affirmative, full plan0.720

At two tokens, the selector's plan is stronger (0.640 vs 0.270). Across all matched-round conditions the selector never exceeds 0.640, while the fixed affirmative plan reaches 0.720 at full length. Plan choice matters mainly for short openings; longer affirmative injection raises success. The sweep is in Sec. 5.2 and Appendix A.1.

W2 + Q2 (scope of the safety claim, and A.8). We accept both criticisms: the abstract says "probability-threshold acceptance held every attack we construct at the floor, including a rule-aware adaptive search", with Proposition 1 and A.11 limits stated. On greedy drafting, you are right: a greedy draft puts all probability on one token, so rejection sampling still preserves the target distribution [1, 2]; A.8 narrows to decoupled proposals.

We also ran the adaptive attack you asked for on Llama-3-8B.

RuleASR (n=100)
typical δ=.10.000
typical δ=.050.000
fixed c=.010.000
top-k=20, same prefixes0.000

The search (beam 3, depth 6; every token must pass the rule) commits its prefixes almost fully (5.0 of 6 tokens), yet 65 of 100 outputs are refusal openers ("I cannot provide..."): threshold-passing tokens are target-likely, and the target's likely continuations of harmful requests are refusals. The top-k control shows they carry no commitment. We did not train a draft end-to-end against the rule. Search is the stronger test: Proposition 1 caps any draft, trained or not, at cutoff-passing tokens, exactly the set our search explored, and it held only refusal openers; a trained draft could still exploit patterns our scoring missed (both in A.11).

W3 + Q3 (speed, quality, and attack in one configuration). One configuration: frozen loop (Llama-3-8B, 1B draft), one GPU, batch 1; benign quality: 220 public instructions (MT-Bench, Alpaca), Llama-3-8B judge; attack: poisoned draft, same config (n=60).

Ruletok/sQuality 1-10Win rate (blind)*Attack ASR
exact34.18.59ref0.017
typical δ=.141.38.520.480.000
hybrid (top-20 and p≥.01)44.0 (+29%)8.480.560.000
top-k=2046.2 (+35%)8.370.550.833

*vs exact, ties excluded.

Top-k buys real speed without hurting benign answers; the same setting carries the attack. The hybrid keeps a 29% speedup at zero attack; its 11 to 12% acceptance-length cost in Table 9 is 1.5% wall-clock on the matched attack cells (n=60; 46.7 to 46.0 tok/s). EAGLE and Medusa stay as in Sec. 5.4, one configuration each; Medusa's degenerate outputs are detected and excluded, the rest preserves the direction (Appendix). New table in Appendix A.10, linked from Sec. 5.3. On DFlash we agree: "saves little time" is scoped to same-vocabulary pairs [3].

W4 + Q4 (first-L exact vs an adaptive attacker). Sweep: first L output positions verified exactly, rest under top-k (n=100/cell, SE≈0.05, single seed; speed at L=2,4,8). Attackers: anchor; greedy-walk (commits the target's greedy tokens, then injects); delayed at L. Over all 24 openings the longest greedy run is one token ("I").

LAnchorGreedy-walkDelayedSpeed cost
10.710n/a0.010~0
20.0100.0100.010−5.6%
40.0100.0100.010−9.5%
80.0100.0100.010−10.8%

L=1 fails as Sec. 5.6 reports ("I" passes the check). At L=2 all three attackers fall to 0.010 (no-attack floor, no full compliance) and stay there to L=8. Greedy-walk fails: two greedy tokens already encode a refusal; delayed fails as the start-2/start-3 probes predict (0.005 and 0.010). The revision adds this table and rewrites Sec. 5.6 around L=2: verify two positions, keep 94% of speed. With the hybrid (Sec. 5.3), the two acceptance-only fixes we recommend.

References

[1] Leviathan et al. Fast Inference from Transformers via Speculative Decoding. ICML 2023.

[2] vLLM v0.30.0, greedy-draft acceptance, 2026.

[3] Chen et al. DFlash: Block Diffusion for Flash Speculative Decoding. arXiv:2602.06036, 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the discussion

Dear AC and reviewers,

Thank you for the review period. We summarize the evidence produced during the discussion.

What the discussion produced. Six new experiments, plus one extension run during the discussion period, all in the revision and marked in blue:

  1. Matched-depth sweep: selection helps only at short openings (0.640 vs 0.270); the fixed affirmative plan reaches 0.720 at full length. This answers Reviewer eNUM Q1.
  2. Rule-aware adaptive search under threshold rules: ASR 0.000 across typical δ=.1, δ=.05, and fixed c=.01. This answers eNUM W2+Q2.
  3. One serving configuration for speed, quality, and attack: top-k gains 35% speed at 0.833 ASR; the hybrid keeps 29% speed at 0.000. This answers eNUM W3+Q3.
  4. First-L exact sweep: verifying two output positions holds all three adaptive attackers at the clean floor, at a 5.6% speed cost. This answers eNUM W4+Q4.
  5. Temperature and top-p sweep: the attack holds (0.640 to 0.760); threshold rules stay at zero. This answers HweA Q2.
  6. Second-family weights-only attack (Qwen2.5-0.5B draft, 7B target): 0.380 under top-50, 0.000 under exact and typical. This answers HweA W1.
  7. During the discussion, we extended this attack to the full AdvBench pool, n=519: ASR 0.441. The no-attack floor is 0.080. The intervals are disjoint. The first 100 instructions reproduce the earlier cell label for label.

Where each reviewer stands.

Reviewer 9vqS (weak accept) writes that the revisions address most of their concerns and strengthen the paper. Their stated scope matches how the revision is written, by design.

Reviewer HweA (weak rejection) maintained their rating. Their two remaining points were statistical reliability and deployment realism. The n=519 result answers the first. The 4B to 14B span of the main grid answers the second, together with documented draft-supply interfaces in production stacks.

Reviewer eNUM (weak rejection) did not participate in the discussion. The four concerns in their review are resolved by items 1 to 4 and by the revision's corrections.

We believe the concerns are resolved with measured evidence, with boundaries stated honestly. Thank you again for the review period.

The Authors (Submission 21281)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.