- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies independent multi-head drafters at depth two, assuming that the first-position distribution matches the target's. It proves an exact decomposition of joint-distribution error into second-position marginal error and a recoverable dependence term. Experiments on Medusa, Meta MTP, synthetic perturbations, and trained heads show that marginal error strongly predicts, but does not determine, how much introducing dependence can recover.
Critical strengths
- Under the exact-first-marginal assumption, the paper derives an exact decomposition separating second-marginal error from recoverable dependence, enabling diagnosis without training another drafter.
Critical weaknesses
- The main result is restricted to two tokens, an exact first marginal, and fixed position-wise marginals. The paper does not establish an extension to longer draft blocks or analyze its computational cost.
Questions for authors
-
Can the recoverable-dependence bound derived for two-token drafts bound or predict gains for longer draft blocks?
-
Is there evidence that this metric helps select effective drafter modifications? In particular, how should it be used when a modification changes the marginals as well as the dependence
I appreciate the authors’ efforts in the rebuttal and revision. The new results address most of my concerns and I have increased my rating accordingly. However, I would have preferred the key additions to be more smoothly integrated into the main text to support a clearer and more coherent narrative, rather than presented in lengthy appendices. In addition, I strongly encourage the authors to improve the paper’s overall readability.