- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies speculative decoding through a controlled empirical framework called FACT, focusing on three factors: (i) drafter training, (ii) candidate allocation under a fixed depth-wise budget, and (iii) execution conditions. The setup uses a lightweight feature-prediction drafter with a frozen Llama-3-8B-Instruct target under greedy decoding.
Critical strengths
The empirical study is relatively controlled and transparent compared with many systems papers in this space. I appreciated that the paper repeatedly states what is held fixed, what is not, and what conclusions the data can support. The transfer experiment to the official EAGLE drafter is also a worthwhile check, since it shows the allocation effect is not restricted to one custom-trained checkpoint.
Critical weaknesses
- The paper’s novelty is limited, and the main positive results come from recombining or re-evaluating known ideas rather than introducing a clearly new algorithmic insight. The training ingredients are largely drawn from prior work (rollout exposure, decayed step weighting, cross-step gradient flow), and the allocation side is also explicitly a restricted variant of prior dynamic tree selection ideas. I do not think the resulting empirical insight is strong enough to compensate for the limited methodological novelty.
- Several central conclusions are not cleanly attributed because the comparisons confound multiple factors or use mismatched checkpoints/settings. The RQ1 result compares “uniform detached rollout” against “step-decayed full-gradient rollout,” which changes both weighting and gradient flow at once. The paper acknowledges this, but the conclusion is still presented prominently. Likewise, the stronger drafter used in the allocation section is trained for 20k steps and is not the same checkpoint family as in RQ1, making it hard to connect the training and allocation narratives into a single coherent claim. In the end, the paper argues for joint evaluation, but the study itself does not always isolate the interactions cleanly enough to support strong takeaways.
Questions for authors
- The main RQ1 result does not isolate step weighting from cross-step gradient flow. Please clarify what conclusion is supported beyond the combined comparison.
- Please report a default-setting cost breakdown for static vs. dynamic decoding, including tree construction/selection overhead, to support the claim that dynamic allocation improves latency under fixed budgets.
- The study is restricted to greedy decoding, one target model, and one hardware platform. Please clarify the scope of the claimed conclusions and which statements should be read as setting-specific.
- Please clarify what exactly is concluded from the proxy-loss ranking failure, given that the proxy is based on an intermediate-feature readout rather than the full target distribution.