- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This manuscript introduces MISTLETOE, a mechanism-level attack against speculative decoding. The proposed method combines a draft-token surprisal objective, a KL-based semantic-preservation objective, and a null-space projected optimization procedure.
Critical strengths
-
The paper is well organized and easy to read.
-
Experimental results show consistent degradation of speculative decoding efficiency across multiple models and decoding frameworks.
Critical weaknesses
-
The evaluation is conducted on a relatively small subset of some benchmarks (e.g., 100 randomly sampled examples from HumanEval and GSM8K). Since the reported conclusions are based on a single sampled subset, it is difficult to assess the stability of the results with respect to sampling variability. I encourage the authors to evaluate additional random subsets (or the full dataset) or report variance statistics to strengthen confidence in the results.
-
I am not fully convinced that this is a distinct security vulnerability. For example, the paper does not compare against random suffixes, noisy natural-language suffixes, or existing adversarial prompting methods.
-
A key claim of the paper is that the proposed method works by collapsing draft-token acceptance. However, most of the reported results focus on speedup and average accepted length. While these metrics are related, I expected more direct measurements, such as token-level acceptance rates, rejection rates, or verifier fallback statistics.
Questions for authors
-
How much stronger is MISTLETOE compared to simpler perturbation strategies?
-
Can the authors provide more direct evidence that acceptance collapse is the primary mechanism?
-
How sensitive are the results to dataset sampling?