acceptodds
Under review as a conference paper at ICLR 2027

Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding

Abstract

Speculative decoding is widely used to accelerate large language model (LLM) inference by drafting multiple candidate tokens and verifying them with a target model in parallel. Its efficiency critically depends on the average accepted length τ , i.e., how many draft tokens survive each verification step. In this work, we identify a mechanism-level vulnerability in model-based speculative decoding: the drafter approximates the target model distribution, but this approximation is inevitably imperfect. Such a drafter–target mismatch creates an attack surface where small perturbations can preserve the target model’s visible behavior while substantially reducing draft-token acceptability. We propose MISTLETOE, a stealthy accelerationcollapse attack against speculative decoding. It jointly optimizes a degradation objective that decreases drafter–target agreement and a semantic-preservation objective that constrains the target model’s output distribution. To resolve the conflict between these objectives, we introduce a null-space projection mechanism that projects degradation gradients away from the local semantic-preserving direction, suppressing draft acceptance while minimizing semantic drift. Experiments across multiple target models and speculative decoding systems show that MISTLETOE substantially reduces average accepted length τ , collapses speedup, and lowers token throughput, while largely preserving task-level performance. Our work highlights a mechanism-level attack surface in speculative decoding beyond existing output robustness, motivating more robust LLM acceleration systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
speculative decoding, speed-up collapse, stealthy attack

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_4gzzOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript introduces MISTLETOE, a mechanism-level attack against speculative decoding. The proposed method combines a draft-token surprisal objective, a KL-based semantic-preservation objective, and a null-space projected optimization procedure.

Critical strengths
  1. The paper is well organized and easy to read.

  2. Experimental results show consistent degradation of speculative decoding efficiency across multiple models and decoding frameworks.

Critical weaknesses
  1. The evaluation is conducted on a relatively small subset of some benchmarks (e.g., 100 randomly sampled examples from HumanEval and GSM8K). Since the reported conclusions are based on a single sampled subset, it is difficult to assess the stability of the results with respect to sampling variability. I encourage the authors to evaluate additional random subsets (or the full dataset) or report variance statistics to strengthen confidence in the results.

  2. I am not fully convinced that this is a distinct security vulnerability. For example, the paper does not compare against random suffixes, noisy natural-language suffixes, or existing adversarial prompting methods.

  3. A key claim of the paper is that the proposed method works by collapsing draft-token acceptance. However, most of the reported results focus on speedup and average accepted length. While these metrics are related, I expected more direct measurements, such as token-level acceptance rates, rejection rates, or verifier fallback statistics.

Questions for authors
  1. How much stronger is MISTLETOE compared to simpler perturbation strategies?

  2. Can the authors provide more direct evidence that acceptance collapse is the primary mechanism?

  3. How sensitive are the results to dataset sampling?

Reviewer_VkZNOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Summary of contribution: The paper studies an attack on speculative decoding. The goal is to somehow reduce the efficiency of SD by modifying the prompt. The main objective in constructing the prompt modification is that it should lead the "drafter" to produce unacceptable tokens (unacceptable by the larger "target" model) while allowing the target to still perform well when taking in the modified prompt. This will evidently reduce the # of accepted tokens by the target, thereby leading to successful attack on the efficiency of SD.

Critical strengths
  1. The paper studies the robustness of speculative decoding by examining how input perturbations can affect the efficiency of SD while leaving the target model’s responses largely useful.

  2. The proposed method is structured around two objectives: reducing draft-token acceptance and limiting changes to the target model’s predictions. The proposed gradient updates and KL-based methods are reasonable.

  3. Authors show experiments spanning multiple target models and speculative decoding methods, with ablations and transfer experiments.

Critical weaknesses

My biggest concern is the motivation for studying this problem and the practical justification for the threat model. These are central to my assessment: the paper needs to establish why the proposed attack matters and who is meaningfully harmed by it. Without a convincing explanation of this significance, the technical results are difficult to place in context. Let me explain the two key concerns:

  1. Operational motivation for the threat model. The main attack assumes white-box access to both the drafter and target models. Although Section 3.2 describes a malicious client submitting adversarially modified prompts, the practical setting underlying these access assumptions remains insufficiently developed. In what deployment would such a client have access to both models for attack construction, and how would this access relate to the actual serving configuration? The transfer experiments partially address this concern, but the authors should connect these assumptions and results to a concrete, plausible deployment scenario.

  2. Significance of the resulting efficiency degradation. Even granting the assumed model access (which I am not convinced with), it remains unclear why this attack presents a consequential practical problem. The attacker submits their own prompts, so the immediate effect is to make their own requests slower and more expensive to process. Even in the extreme case where no draft tokens are accepted, the target model can continue generating the response while the system loses its speculative acceleration. The paper needs to explain when this loss translates into meaningful harm to the LLM service operator or other users using this service. The energy and throughput measurements in Appendix C are useful, but they do not establish whether an attacker can impose substantial costs or service degradation with relatively little effort. For example, can a modest volume of malicious requests significantly affect legitimate users, and can the cost of constructing the attack be amortized through reuse?

In this reviewers' opinion, any new threat model or scenario should be accompanied with a solid justification (beyond the assumptions) with practical and operational significance. This paper clearly lacks those.

Questions for authors

See my comments for weaknesses. If you can provide a clear justification/relevance for studying this problem, I am willing to reconsider my score.

Reviewer_Cj3SOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The author proposes an adversarial prompt attack that degrades speculative decoding efficiency while limiting changes to the target model’s responses. It optimizes a per-prompt suffix to increase drafter–target disagreement, using a KL-based preservation constraint and gradient projection to reduce draft-token acceptance. Experiments across multiple models, speculative decoding methods, and benchmarks demonstrate reduced decoding speedups.

Critical strengths

Overall writing is clear.

Critical weaknesses

The practical motivation for this attack is not yet clear to me. The reasons are as follows

  1. Attacker cost versus provider cost. The current per-prompt optimization makes the economic motivation questionable. If the attacker performs up to 50 optimization iterations before submitting a query, the attack-generation cost could exceed the additional inference cost imposed on the provider. A stronger demonstration would find a single universal suffix that generalizes across unseen prompts, allowing the optimization cost to be amortized. Otherwise, the paper should explicitly quantify attack-generation costs and establish how much reuse is needed to break even. In another word, can this method be extended to universal attack (like universal adversarial patch that we had before; for adversarial attack, per prompt optimization can be motivating since, it aims safety issue, but for this paper, per prompt attack does not make sense since we are discussing efficiency).

  2. Magnitude of the actual damage. A substantial reduction in speculative speedup does not necessarily imply substantial economic harm. If the main effect is to bring latency back toward standard autoregressive decoding, with some additional FLOPs from rejected drafts, it is unclear how damaging even a successful attack would be. Frankly speaking, I am not really motivated by the problem scope.

  3. Transferability to an unknown serving system. To be realistic, we need to assume that we don't have any knowledge of the company (i.e., which model they are using or which SD method they are using). So to provide realistic setup, the authors need to consider finding a prompt in attacker system and test in on different model and SD method. Especially, many recent papers use diffusion based SD (not RNN or small transformer in the head). I do agree the author partially resolved this (in Table 5 and Appendix), but more experiments (e.g., using diffusion based SD or to larger model size would be appreciated).

Questions for authors

See the questions above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.