- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AdaptSpec, which employs reinforcement learning (RL) to jointly optimize both SL and CL. The experimental analysis identifies several influential factors, including input categories, that affect the configuration yielding the highest speedup. The authors conduct experiments across four draft–target model pairs and five datasets.
Critical strengths
The paper provides an experimental analysis of several potentially useful signals, such as logit divergence and step-wise acceptance ratios, which could help guide adaptive decisions and improve the accepted length.
Critical weaknesses
1 Insufficient coverage of related work and outdated baselines. The paper lacks citations to several important and closely related studies, and the baselines used for comparison are relatively outdated. SpecInfer, which employs tree-based speculative decoding, is an early work in this area. However, EAGLE-2[1], an important baseline for dynamic tree-based speculative decoding, is only briefly mentioned in the Related Work section and is not included in the experimental comparisons.
Furthermore, numerous existing studies have explored dynamically adapting speculative decoding configurations based on different input signals, some of which are closely related to those considered in this paper. For example, previous methods have investigated entropy [2] and average acceptance rates [3] as signals for adaptive decisions, but these relevant works are not adequately discussed or cited.
In addition, several prior studies have explored reinforcement learning for optimizing speculative decoding and tree construction[4], yet these methods are neither sufficiently discussed nor compared against. The authors should provide a more comprehensive discussion of related work and include stronger, more recent baselines to better demonstrate the novelty and effectiveness of the proposed approach.
[1] EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
[2] Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models.
[3] Dynamic Depth Decoding: Faster Speculative Decoding for LLMs.
[4] Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning.
2 Lack of ablation studies on the reinforcement learning design. The paper does not provide sufficient ablation studies to justify its reinforcement learning framework. For example, the reward function design is not thoroughly investigated, and comparisons with alternative reward formulations are missing. These ablations would help clarify the effectiveness of the proposed RL framework and the contributions of its individual design choices.
Questions for authors
Please refer to the questions raised in the Weaknesses section.