- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces UBTree, a speculative decoding technique that combines the efficiency of parallel drafting with dependency-aware tree construction. A parallel drafter first proposes multiple per-position unigrams in parallel. Then, the bigram selector identifies the plausibility of the bigrams to effectively construct tree drafts from the unigrams. The paper provides broad experiments to showcase the effectiveness, spanning over multiple models, benchmarks, and serving scenarios, where UBTree provides consistent and significant speedups.
Critical strengths
The motivation of the paper is strong and agreeable (naive tree construction from unigrams would likely be suboptimal in terms of high-quality draft tree construction), and the proposed parallel bigram selection approach directly and effectively addresses this problem. The proposed approaches, including both tree construction and tree-aware training, leads to significant empirical performance gains. The gains are consistent over an extremely broad range of evaluation, suggesting the significant practical value of the paper.
Critical weaknesses
I did not find any critical weakness.
Questions for authors
I do not have any further questions that may materially affect my assessment.
We sincerely thank the reviewer for the positive assessment.