- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the lack of within-block causal conditioning in parallel speculative drafting, which limits the acceptance length of DFlash-style drafters. Motivated by the observation that later draft tokens can benefit from conditioning on previously predicted tokens, this paper proposes Branch-Jacobi, a multi-pass refinement framework that uses packed prefix-conditioned branches and AR-path consistency checks to progressively refine draft proposals. It further introduces moving-anchor training and causal prefix supervision to improve the drafter's ability to handle variable-length prefixes. Experiments on Qwen3-4B and Qwen3-8B across seven benchmarks demonstrate 4.93–6.40× decoding speedups over autoregressive decoding, outperforming DFlash by 25.3–29.2% while improving average acceptance length.
Critical strengths
The paper demonstrates substantial and consistent improvements in decoding efficiency across seven benchmarks on Qwen3-4B and Qwen3-8B. Branch-Jacobi achieves 25.3–29.2% higher speedups than DFlash while improving average acceptance length, demonstrating the effectiveness of the proposed approach.
Critical weaknesses
-
The core motivation of this paper has been extensively explored in recent studies, such as DSpark, xPress, and DFlash 2, which similarly aim to improve parallel speculative drafting by recovering within-block token dependencies. Although Branch-Jacobi introduces a different refinement mechanism, its conceptual novelty remains a bit limited. Furthermore, the lack of direct experimental comparisons with these closely related methods makes it difficult to assess the significance of the proposed design.
-
Considering the computational resources required for training, the evaluation on Qwen3-4B and Qwen3-8B is understandable. However, the paper lacks direct experimental comparisons with closely related methods, such as DSpark and xPress. At least one representative comparison under comparable experimental settings is necessary to establish whether the proposed refinement mechanism offers meaningful advantages over existing approaches.
If the paper provides stronger evidence of its advantages over existing methods, I may reconsider my rating.
Questions for authors
-
What are the key conceptual and technical advantages of Branch-Jacobi over closely related methods such as xPress,DSpark or Domino?
-
In Table 1, for Qwen3-4B at T=0, the average speedup decreases from 6.35× at F=3 to 6.11× at F=4, but slightly increases to 6.13× at F=5. What explains this non-monotonic behavior, and how can the optimal number of refinement passes be determined across different tasks and model configurations?
5 comments
A Gentle Follow-up on Our Revisions
Dear Reviewer Zvvw,
We appreciate your constructive feedback on the technical contribution and refinement budget. Following your comments, we have revised the manuscript with clearer distinctions from related methods, comparisons against recent baselines, and an extended pass-budget study. If you have time, could you please let us know if any concerns remain?
Best regards,
Authors of Branch-Jacobi
As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.
Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.
Thank you all for your time and constructive engagement.
Response to Reviewer Zvvw (Part 1 / 1)
W.1–2 / Q.1: Technical Contributions and Recent Baselines
We thank you for your careful review and for recognizing the substantial and consistent improvements in decoding speed and acceptance over DFlash across seven benchmarks on Qwen3-4B and Qwen3-8B.
Thank you for asking us to clarify the technical contribution. On Qwen3-4B at T=0, with the same 100K training examples, Branch-Jacobi (F=3) achieves a seven-benchmark geometric-mean speedup of 4.72×, compared with 4.30× for DSpark, the fastest recent baseline.
Relative to DSpark, Domino, DFlash2, and XPress, Branch-Jacobi reruns the same drafter backbone on packed moving-prefix rows to recompute suffix representations. First-MASK checks identify the current proposal's common-prefix boundary with the fixed greedy drafter-AR reference and select one already-computed complete branch for a single linear target verification. General Response #1 provides the mechanism distinctions and direct comparisons.
Moving-anchor training and causal prefix supervision adapt the drafter to recomputation under different prefix lengths, providing a training design matched to this inference procedure.
The theoretical guarantee concerns prefix consistency with the drafter path, rather than target acceptance of the entire suffix. In the separate complete-branch versus first-MASK assembly comparison (pKWA W2; corresponding table in the revised Appendix), we use the same drafter checkpoint and packed-branch evaluation protocol, changing only the proposal-update rule while keeping target verification unchanged. Based on these tabulated values, complete-branch selection has higher reported speedup in all 24 configurations, with relative differences of 4.1%–26.3%. These results provide empirical evidence concerning the update design alongside the recent-baseline comparisons.
Q.2: Refinement Pass Budget
Thank you for pointing out the non-monotonic trend in Table 1 and asking about the choice of F. General Response #2 reports our timing re-evaluation of the submitted Tables 1, 3, and 4 and updated measurements. For Qwen3-4B at T=0, recomputing the F=4 seven-benchmark geometric mean from the reported benchmark values gives approximately 6.2×, between 6.35× at F=3 and 6.13× at F=5. The updated aggregate therefore no longer shows the F=4-to-F=5 rebound seen in the submitted table. The corresponding aggregate AAL values remain 8.27, 8.41, and 8.45.
Choosing F. Higher AAL alone does not determine the preferred budget; we select F using measured response-mean TPOT. We recommend F=3 as a practical default for the main-table settings, with a small latency sweep for further task-specific optimization. The preferred budget also depends on serving load: our concurrency sweep favors F=3 at C=1–4 and F=2 at C=8–16.
Execution cost and extended pass-budget sweep. The fixed-shape implementation in Appendix D executes all scheduled draft forwards, including after certification is complete; extra passes therefore continue to incur draft computation.
To check whether the decline in seven-benchmark geometric-mean speedup observed from F=3 to F=5 continues at larger budgets, we extended the sweep to F=6 and F=7 using the same Qwen3-4B checkpoint and timing protocol as Table 1, at T=0 and block size 16. Results are reported as mean ± sample standard deviation over four runs with a shared AR baseline; other reporting conventions follow Table 1. The full results of the extended sweep have been added to Appendix E of the revised manuscript.
Average acceptance length remains approximately 8.46, while average speedup falls from 5.98±0.07× to 5.84±0.03×. These measurements illustrate diminishing returns: additional draft forwards incur computation while yielding little further acceptance gain.