- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Parallel Reconsideration for feature-level speculative drafters. A learned self-query lets the drafter revisit a feature without advancing to the next token, while a learned halting gate controls further reconsideration. Reconsideration rows are batched with ordinary draft steps, and the resulting candidates compete within a fixed verification-tree budget. Training combines the ordinary prediction loss, depth-weighted reconsideration losses, a gate objective, and mixed-depth caches. Evaluation on three target models and seven benchmarks reports average accepted-length improvements of 5.3–10.2% and relative speedup improvements of 3.5–9.2% over EAGLE-3.1. Target-only tree verification is unchanged.
Critical strengths
-
The method offers a concrete way to spend additional computation on uncertain draft features without adding a separate sequential draft step. The self-query, mixed-cache training, and serving algorithm address the distinction between revising the current prediction and advancing its continuation. Appendix C.4 also explains why changing the candidate tree does not change the target output distribution under the specified verifier.
-
The experimental controls are stronger than a comparison at equal verified-node counts alone. The paper retrains component ablations, compares against a two-layer drafter, and gives EAGLE-3.1 the same additional rows for wider drafting. Table 4’s PR-without-reconsideration control is particularly useful for separating the effect of the new training recipe from the effect of actually reconsidering at inference. The main results cover multiple models, domains, temperatures, and request batch sizes within the same serving engine.
Critical weaknesses
-
The latency measurements do not fully support the broad claim of negligible drafting overhead. Appendix H reports a whole-cycle ratio of 1.015–1.022 on Qwen3, combining drafting and target verification; it does not measure the increase in drafting time separately. All wall-clock results use one H200. Table 2 also shows cases where PR is slower than EAGLE-3.1, including Llama LiveCodeBench at request batch sizes 8 and 16. The paper should specify when the additional computation remains cheap and explain these regressions.
-
The paper does not report variation across independently trained drafters or repeated timing measurements. Several individual benchmark gains are small, and the component ablations cover selected configurations. Since the method changes both training and inference, it is unclear how consistently the gains would appear across training runs.
Questions for authors
-
Can you separate draft-model execution, reconsideration bookkeeping, and target verification time for the existing batch-size experiments? What explains the Llama LiveCodeBench regressions at batch sizes 8 and 16?
-
How many independent drafter-training and timing runs underlie the main table and the equal-row controls? If repeated runs are available, please report their variation and the additional training cost relative to EAGLE-3.1.